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SOME SHORT TERM EFFECTS OF PROJECT HEAD START: 
A PRELIMINARY REPORT ON THE SECOND YEAR OF 
PLANNED VARIATION — 1970-71 

I. INTRODUCTION 

Background 

During the early months of 1969 the Office of Child 
Development planned a three wave longitudinal study de- 
signed to assess the relative impacts of a variety of 
preschool curricula. , The study was called Head Start 
Planned Variation (HSPV) and began in fall 1969. Plans 
called for the systematic assignment of-<^^umber of well- 
developed' curricula, each to two or more sites throughout 
' , the country. Selected sites were to meet three criteria. 

First, each site was to contain an on-going Head 
Start Program. No funds were allocated for serving chil- 
dren other than those already being served by Head Start. 

Second, each site was to draw participant children 
from a preschool population living largely within the 
attendance area of a school or schools where older chil- 
dren attended a Follow-Through program.* By fall 1969 



* Follow-Through is an intensive early elementary (K-3) com 
pensatory program designed to enrich the experiences of eco 
noroically poor children — particularly poor children who 
have had Head Start experiences. Originally intended to be 
a national program, Follow-Through was designated as an ex- 
perimental effort in 1968, one year after it was initiated. 
By 1969 there were over 170 school districts with Follow- 
Through programs. 
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most Follow-Through schools had adopted one of a nuirODer of 
wcll-^defined educational curricala. These programs were 
being evaluated by the Office of Education. Children 
entering selected Follow-Through schools during the years 
X969-1972 were to be tested at entrance and longitudinally 
followed and tested until they completed Follow-Through at 
the end of third grade. 

Third, the selected Head Start site had to agree to 
adopt the curriculum model beina used in tbf* Follow-Through 
schools in its area. Aid in implementing the models was 
to be provided by consultants responsible^ to the original 
architects of the models. In additionp-e^tra funds for 
purchasing equipment and for hiring teacher aides were to 
be provided to the selected Head Start classes. Overall r 
the cost of implementing the Planned Variations model is 
estimated to be $350.00 per child above the cost of con- 
ventional Head Start (see McMeekin, forthcoming , for a 
detailed estimate of the extra costs). Since many of the 
Follow-Through curricula were adopted from programs origin- 
ally designed for pre-schools, the use of them in Head 
Start programs was appropriate. 

Sixteen sites were selected for inclusion in the 
Planned Variations study during the 1969-70 school year. 
Eight curriculum models were represented, each by two sites , 
and formed the sample for the first wave of the study. The 
second wave of the study, the school year 1970-71, included 



-3" 



thirty-seven sites. Since one of the original 16 sites had 
dropped out of the study, this meant that 22 new Planned 
Variation sites were added in 1970-71. Fourteen of the 22 
new sites followed one of the original eight models and 
were located in a Follov;-Through area. Of the remaining 
eight new sites three were in Follow-Through locations (one 
in each of three models) and five were located in sites 
without Follow-Through schools. The final five followed a 
curriculum designed by the parents and staff of the chil- 
dren -in the site in collaboration with a consultant from 
the Office of Child Development. The third wave of the 
study (1971-72) involved the same sites as the second wave 
with two exceptions: two sites were dropped and one was 
added. 

The design of the Planned Variations study called for 
children in all three waves to be tested at the beginning 
and end of their Head Start experience. Following Head 
Start, the children would enter the Follow-Through program 
in their community and be evaluated at the beginning and 
throughout their Follow-Through experience. The records 
of the Head Start and Follow-Through testings could then 
be linked. The linkage would provide data for a longitud- 
inal assessment of the combined pre-school and early ele- 

* 

mentary experiences of the Planned Variation children. 

Testing was also planned for other groups of Head 
Start children in every Planned Variation site. These chil- 

Er|c )'hio7 



dran would attend Heau Start classct; without a designated 
curriculum component and serve as a local comparispn group 
for the study of the Planned Variations. Head Startj classes. 
Witli some exceptions this strategy was follov;ed fc{)r all 
waves of the Planned VariatiQns study. The compa^jrison 
children v;ere also to be included in the Follov7--Through 
evaluation. 

Progress reports on Planned Variations were |;)lanned 
at three tiir.es during the course of the study: first at 

the end of the Head Start experience for each of the three 

i 

vraves of children; second, at various times durindj the 
Follow-Through experiences of the three waves; and third, 
in 1976, after the third wave of Head Start children had 
completed Follow-Through. A preliminary report on the 
first wave (1969-70) was prepared in 1971 by the Stanford 
Research Institute for the Office of Child Development. 

The present report is one of a set of preliminary 
reports on the second wave of Head Start Planned Variations 
Final reports, to be available in September 1973, will 
review the one-year data in all three waves of Planned 
Variations . 

Purpose of the Report 

This report attempts to ansv;er three questions: 

1. What are the short term effects of a Head Start 
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Gxperience on children? 

2, Are there discernable differences between the effects 
cn children of a Head Start Planned Variations cxper-- 
ience and a conventional Head Start experierce? 

3. Do Planned Variation ir^odels differ in their effects 
on Head Start children? 

In all instances the measured effects we discuss here are 
narrowly defined. Specifically, we are concerned with three 
measures of cognitive achievement^ one measure of intelli- 
gence and one measure of motor control. No attempt is made 
to introduce data about the many other areas which a pre-- 
school experience might influence. 

This report has been prepared in conjvmction with three 
other reports about Wave Tv/o of Planned Variations, One 
report considers the process and success of implementing the 
Planned Variation curricula in the various sites. A second 
report presents a detailed smrjnary of the various measuring 
\ .instruments used in all three waves of the Planned Variations 

study. A third report explores the. possibility that differ- 
ent characteristics of children interact with particular curri- 
cula to produce different results. This final report ana- 
lyzes children in both Waves One and Two of the stiidy." 

No report in this series attempts an overall systematic 
review of the preschool literature. For this the interested 
reader should see Datta (1971) , Stearns (1972) or White et 
al. (1972) . And no report in the series attempts to provide 

ER|c HH.,}(j 
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a detailed description of the twelve Planned Variation models-. 
For this the reader should aoc Kaccoby and Zellner (1971) , and 
the Rainbow sc-riea published by the Office of Child Deve- 
lopment (1972) 

Limitations o^thiiiJ^ort 

It is impossible for a report of this nature to capture 
the richness and corp.plexity of a child's Head- Start experience. 
The best we can do is to report feuininary estimates for a 
very narrow range of effects of presumably different pre- 
school experiences. Four Specific constraints on the value 
of this study should be not^d at the outset. 

1. Like almost all studies of school effects, we assume 
a production model oA' the preschool process. An 
analysis of this nature requires us to initially 
measure certain inputs of the child and his class- 
room which we think may be important, make assump- 
tions about the homogeneity of children's experi- 
ences within a given, classroom, and then gather 

. some output 3t^ea^uremerits_.on. the ch^ the end 

of his preschool experience. Then, after controlling 
for relevant initial differences among children, we 
compare groups of children on our output measures. 

0 

For the most part we make no attempt to understand 
the diversity of experiences that children bring or 
ERsLC" have in their preschools. One reason for the 



narrovmess o£ our approach is the lack of a consistent 
theory of child dev^,loprent,v another reason for it is 
the lack of a strategy of analysis which is sufficiently 
complex to deal with moxe than the skelaton of reality, 

2. The lack of a consistent theory of child development 
is reflected in the sparsity and limitations of the 
measures used in this stxjdy* As we noted earlier we 
will report only on four ir^easures of tBe' cognitive 
area and one measure of motor control** Though these 
measures are among the best to be found/ they still 
have only questionable validity, (See Chapter 2 and 
the report **The Quality of the Data^^*) 

3, In order to justify comparisons among curricula we 
have to inake assumptions abiput the integrity of the 
various curricula in different sites. The initial 
assumptions are: first, that the various preschool 
curricula do create discernably different class-- 
room environments; and second, that the curricula 

are exportable — that is, that they can be implemer.ted 
in various classrooms around the country. These 



* Two other child output measures v;ere included in the 1970--71 
data collection. The reasons for their exclusion here are 
outlined in Chapter 2» 
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assuj^ptionn ara discussed in th^ report en 'U:^^pU:-" 
imfmi^ai.i:on xr. Planned Variat^^n.; B7d-?V'* 
that report points out, they ar^ i^either trivi,al nor 
alvay.n valid. By and large, h^wavc^r^ plunge ahead 
and ^accop't then valid ^ for given th^s^ assump- 
tion^$ ^/e'lj^xght be ablo to attribute ' dif ferences m 
child outputs to the infli^mncB of the different Mdcrl;^... 
4. Problur^'p also ster fror^^ this study d^niqn. Fitm., 
as aesqribed afcovoi. "Planned V'arxatien S2it^s wcrre 
not rando^nly assigned 'to ]n^odals* Rath^i^'^ the suites 
vmr€- selected on two criteria unrelated to the? 
raquin^msnto for an adequate.* experiin^efital det^iqu^ and 
then givm the opportunity to accept or rejaet the 
assigned curriculur* nomovQt^ thm local ammimitf 
had control over tha specification of which classes 
within a site v^ere to c^rploythe Planr^ed Variations 
{PV) curriculujm* Since th^ selection of cotmari^on 
classes wit!:in the isites occurred ^after 'sc^lecVon of 
the Planned Variation classes^ the treaferae^t CPV) 
and comparison {NPVi elasseis cannot be asfumri to be 
randen samplers draWn fre^ri the sa^e populatioT^. Thus^ 
randoSi¥tSIon"^di "not sceur at €^ithiBt of tte tw^ 
critical" design points at thc^ lovei of asaignpc^nt 
curricula to sit^s or at tha level of assigniwnt of 
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trf3:ati?teni ar.d ccmparxBon groups within site^* 

■ ^%*ifcho-at rande^Liitation wt^ m'^mt mly on statistical 
technique's to control for thQ ittilmna^s of factors 
associated with the sel^ctitm proeesses. Since wci^ do 
not liaw a cle^r undarstanding of tht^ motivations and 
I2^'€*ehanitms which auAded thc^ \'ari©as .■ ap^opg«-site and 
%^ithin-site amiq^mtnt prac€:clures# our approach i^^ust 
be to crer^trt^I as t^-'uch possible the variables which , 
ra^jhfe be- rc? levant* If oiJr dav@icpp.@ntal theory ^^#re 
admp^ate an^J fh€- rr4?-*ti: of replications of each eisrri-* * 
C4^lui^^ large? cnmigh^ non-^raj^cloitu^^tioD inight not be ii 
serieu-s prablc^^^-. But our traiory is not adeqt^ate to 
fully specify all potpihl^. important and uneontrolied 
inf luc^i^ees « And €^v^n if oiir theory ^ere adequate t-^c^^ 
utiiRber o€ repi icat:iiens of Cf^ch curriculism in the stydy 
is probably to^-^ s^r^ail to acccEmodat?;^ all of the noee- 
'Ssary c©ntrol^. The r€^lat:ively fcrw^ replications for, 

{eyrraculai -bt;:<h partj^aily and fully confoundaci 

ablcs* .Tl.ft* lack cf a -fully i'^i:: v^^* I opec ■ theory a^d the conl^^vu^diiig 

> 
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Strenc^ths of the Study 

Granting the above, what particular strengths does this 
study bring to the analysis of the effecta of preschools on 
economicifilly poor children? To answer this we have only to 
loolc at previous research in the area* Three characteristics 
of the Planned Variations study stand out. 

1. There is an attempt in Planned Variations to system-- 
^ atically vary the preschool environments of children 

in a nurfd:)ar of locations around^ the country. Prior 
to thisr national studies of preschool have looked 
only at naturally occurring differences among class- 
room environments. (See Westinghouse-Ohio, 1968 or 
**Tha Study of Natural Variations in Head Start/' 1969. 
There have been studies of systematically varied pre- 
school envir'onments in single locations (see Bissell 
1970 for a s\xynmary) but never before has there been 
a national study of this sort* 

2. We have great confidence in the care and accuracy of 
the'dat^ gathered in this study •.^^-^n'e^ many studies 

, , have gat^^i^rod pre- and post-test data, and information 
on children and teachers 'characteristics, no data 
collection effort for a national study has been as 
;cKiri::*f ully administered and conducted • For a review 
of the data collection procedures see "The Quality of 
; the' Planned Variations Data*** 
, 3./ Thv Planned Variations study, has multiple replications 



i 
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Among waves there is replication of the success of 



\ particular s}.tes and curriculum models, . Within waves 

there are generally two or more sites using the same 
model. Though models were not randomly assigned 
to sites, the fact that both form of replication 
exist serves to greatly increase our confidence in 
the validity of measured effects of the various pre- 
j school models.* As far as we know there is no other 
I study of preschools with a planned strategy of cur- 
/ riculum replication. ^"^^^"^^^ 

Strategy for Analysis 

The strategy for Analysis is dictated in large part by 
the constraints on the study. First, we will focus principally 
on the analysis of cognitive growth. To do otherwise would 
be tp seriously overplay the existing data. In doing this we 
recognize that we are not even attempting to capture the rich- 
ness^ of a preschool experience or the largest part of the 
differences among preschools. 

Second/ we display the data in a more complete manner than 
is normally done. The limitation of theory that we bring to 
the analysis should not foreclose the possibility that other 
people could bring other theories and questions to put to the 

data. Though cumbersome, the \ntent is to let others explore^ 

\ 

their favorite issues. / 



* In fact there is a paradox h^re. V7ere the models randomly ' 
assigned; implying that sites v;ere forced to accept a parti- 
' cular model, V7e might find 'it hard to generalize from the 
results to a situation where sites had a free choice of v;hich 
tl^^ model to choose. HH) ( ^ 
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Third, the lack of a true experimental design puts the 
analysis of the data into a never-never land. Had we random 
assignment of curricula (treatments) to sites, then a compari- 
son of treatments would yield us unbiased estimates. If we 
had random assignment of classes to PV and NPV groups within 
sites, then a comparison of the two sets of classes would 
yield unbiased estimates. If we had two random samples of 

• children from the same population — .one going to Head Start and one 
not, then estimates of the general effects. of a Head Start experience 
would be unbiased. But we have no random assignment, so all 
estimates are biased in some unknown fashion. Estimation of 
effects thus becomes an art instead of a science. There are 
numerous statistical techniques to help *reduce bias (matching, 
covariance., blocking, crossed designs and standardization 
techniques). Each may be helpful depending on the adeg^iacy of 
the structural model we are trying to fit. That is wheire" the 
essential problem lies, for v/e have no a priori way of determin- 
ing which is the best analytic model. Given this state of 
affairs, we follow Tukey's advice: "As in the famous discussion/ 
between Student and Fisher and the interjections by Sir Harold 
Jeffreys, it may not be a bad thing to use all the allowed prdn^^ 
ciples of witchcraft and not just one set." (Tukey, in press,"/* 
p. 112.) ' 

We will not use all of the principles of methodological ^ 

witchcraft but we do use a number. In particular, our strategies 
for removing bias in the data depend on (1) our choice of a 

Er|c I'HmC) 
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statistical model; (2) our choice of variables; and (3) our 
assessment of the accuracy with which the data are measured. 
Different decisions in these areas of judgment lead to a 
variety of estimates of "effects." To some extent tfie vari- 
ability of the estimates will aid in our determination of 
confidence about the magnitude of the effects. Thus the 
variability of results from different analyses gives us a 
sense of confidence ■ limits for the reported effects • Such an 
approach will generally inspire caution in. interpretation, for 
most of the effects found in this study are small. On the 
other hand, large- effects which turn out to be robust — in- 
sensitive to variations in analysis methods — presumably 
should inspire confidence. 

As we note later, estimates of statistical confidence in 
this study are compromised in a number of ways. First, we 
report a large number of comparisons. The effective signifi- 
cance level for any one comparison is thereby reduced. We 
also note that we carried out a larger number of comparisons 
that remain unreported. Second, the variety of methods and 
models used on the same data lead to statistical estimates which 
are not independent of another. Third,' the lack of random 
assignment at any level leads us to make ex post facto argu- 
ments about the representativeness of the data for any given 
population. To the extent that our arguments are inaccurate, 
our inferences to larger populations based on sample estimates 
may be both biased and statistically imprecise. 



Organization of j:he Report 

This report contains seven additional chapters and 
assorted appendicfes. With the exception of Appendix A, all 
appendices are a separate volume. Chapter II sets out 
an overview of the study design, describes the data collection 
instruments and procedures, and contains a brief discussion 

of each of the twelve curriculum models. Chapter III describes 

\ 

characteris^tics of the sample of children, classrooms, an^ 
sites. We consider two issues in detail. First, We explain 

our reason^ for reducing the original totalsampl e of^ oyer 

■ ~ 

6300 childreV to an analysis sample of 2235 children. Second, 
we focus on problems posed by the final analysis sample. 
A number of exemplary tables are included in Chapter HI. 

Chapter IV attempts to estimate the average effects of 
a Head Start experience on children. The procedure used is 
primarily descriptive. Our strategy is three-fold. First 
we present actual gains for various groups of children. Second 
we estimate what v;ould have happened to the scores of children 
had they not been exposed to Head Start. Third we present 
comparisons of the expected to the actual gains for the vari- 
ous groups of children. Appendix B contains supplemental 
tables for this chapter. 

Chapter V discusses a variety of analysis problems. 
First it considers the issue of an appropriate unit of analysis. 
Second, it generally describes strategies for overcoming 
biases in estimating differences among non-randomly selected 
groups. Third it describes in detail two analysis strategies 
ERXCased in Chapters Vi and VII. ,) ^ 



Chapter VI considers the question of differences between 
the effects of Plann'ed Variation Head Start and conventional 
Head Start experiences. To answer this question we disre- 
gard differences among Planried Variations curricula and 
contrast children in PV classes with children in NPV classes. 
Various subsets of these two groups are also contrasted. 
Appendix C supplements Chapter VI with extra tables. 

Chapter VII focuses on the issue of differences among 
curricula. The data for this issue_are__g^J±Lej^@d^-f^^ 
"cedlTig chapters and new summary statistics are generated by 
other analytic techniques. Appendix D supplements 
Chapter VII. - 

Chapter VIII summarizes the preceding discussions. 
Though a summary^ it also pinpcfints the major findings of 
the analyses and raises questions regarding their importance 
and stability. In particular we focus on four major results 
of the study: (1) the magnitude of the overall Estimated 
effects of the Head Start experience; (2) the overall similari 
of the effects among the different programs? (3) the strength 
of one model in imparting specific inf ormat^ion ? and (4) the . 
extraordinary success that one curriculum seems to have in 
raising the IQ level of children. Appendix contains an 
extensive discussion of the fourth result. 



Chapter II 



DESIGN, DATA COLLECTION ACTIVITIES AND 
THE CURRICULUM APPROACHES 



Overview: 




This chaptex--^de^-crj::Bes^ the overall design of the Planned 
Variation Wave Two study. It also includes a brief descrip- 
tion of the measures used in the study and of the 12 curriculiom 
approaches. The next chapter describes the characteristics 
of the children and their classrooms. 

Design : 

Thirty-seven sites had Planned Variation (PV) curricula in 
19 70-71. Twelve curricula (models) were represented. There 
were comparison classes at 14 of the 37 sites (on-site com- 
parison) and at seven locations not having Planned Variation 
classes (off-site comparisons) . Table II-l displays this 
information ♦ Columns 1/ 2 and 3 of Table II-l show the names 
of the twelve curricula, a site code for each site, and the, 
location of the site. The first two digits of the site 
code refer to the model (e.g. all Bank Street sites have 
codes beginning with 05) . The second two digits specify 
the site within the model (Tuskegee is site 0510) . With the - 
exception of the Enabler model, which is unique to the PV 
study, the model and site codes were assignad as part of 
the Follow-Through evaluation and contain no information 



other than identification of model and site. 
: Column 5 of Table II-l contains the year of entry 
of the site into the Planned Variation study. Fifteen of 
the thirty-seven sites were al3o in ^Planned Variation in 
1969. Columns 6 and 7 show the numl^er of classrooms in 

the site. Column 6 shows the number of Planned Variation 

/ - . 

classes. Column 7 shows the numbe^r of comparison classes. 
Blanks in column 6 indicate that the site was an "off-site 
comparison" site. Note that the off-site comparisons are 
paired with Planned Variation sites and are given the same 
site code number as a Planned Variation s4.te. Blanks in 
column 7 indicate that there were no comparison classrooms 
at that si te . 

A few things should be noted from the table. First, 
three of the twelve models have only one site (Pittsburgh, 
REG, and N.Y.U.). Though these models are included in 
analyses when possible, confidence about their effects will 
necessarily be less than for the other models. Since there 
is no site level replication for these models, effects of 
the model and of the specific site cannot be separated. 
Second, there is an uneven distribution of Planned Variation 
and comparison classrooms among the sites. As we point out 
in the next chapter, however, not all of the classes were 
tested — the tested sample levels out the number of class- 
rooms per site. Third, only 14 of the 37 Planned Variation 
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TABLE 

HEAD START COMNtUNITlES 1970-71 



CUHKICliMJM mVV.l 
S!»ONSOU 



Niranicht 
(Far West 
Laboratories) 



Henderson 
(Tucson) 



Bank 
Street 



SITE 
CODH 



02,02 

02.04 
It 

02.05 
02.09 
02.13 



03.08 
If 

0S.Q9. 
05.16 



05.01 

05.10 

05.11 
II 

05.12 



SITH 
COr''>*ijNTTY 



VKSTING 
LEVEL 



Buffalo 
Duluth 
St.. Cloud 
Fresno 
Salt Lake 
Taco;r.a 



I 
II 
III 
III 
I 
II 



LaFayette 
Albany 
Lakewood 
Lincoln 



III 
III 
I 
III 



Boulder 

Tuskcgce 

Wilmington 

DeLaWar 

Elmlra 



ir 

I 
II 
II 
III 



"ThaiTsIte 
joined study 



"7B" 

70 

70' 

70 

69 

70 



69 
69 
70 



"TIT 
69 
69 
69 
70 



NUMBER OP 
PV CLASSES 

— n 



17 

8 
7 



12 
9 



NUMBER OF COM- 
PARISON ClASSnS 



Becker 6 

Englemann 

(Oregon) 



Bushel 1 
(Kansas) 



tfeikart 

(Hl-Scope) 



Gordon 



EDC 



Pittsburgh 



N.Y.U. 



Enablers 



07.03 
07.11 
07.14 



08.02 

08.04 
08.08 

"5or 



E. St. Louis 
Tupelo 
E. Las Vegas, NM 
W. Las Vegas, NM 



III 
III 
II 
II 



"ST 
69 
70 
70 



Oraibi 
Acoiiia 

Portagcville 
MoMnds. 111. 



III 

III 
II 



"ST 
69 
69 

70 



09.04 
09.06 

09^0 



?t. Woitcn Dch. 
Pensacola 
Central Oi 
Greeley 

Seattle^ 



TTT 

III 
I 
III 
II 



69 
t9 

70 
70 



10.01 
10.02 
10.07 
10.10 



Jacksonville 
Jonesboro . 
Chattanooga 
Houston 



I 

III 
III 
II 



69^ 
69 
70 
70 



11.05 
11.06 
11.08 



Washington 
Paterson 
Johnston Co. 



Ill 
IT 
III 



69 
70 
69 



12.03 



20.01 
27.01 



27.04 
27,05 
27.03 
27.02 

27.01 



Lock Haven 
M5f/1 enburg 



III 
III 



70 
70 



Kansas City 



III 



St. Thomas, VI 



BiU-ings 
Colorado Spr. 
Bellows Falls 
Newburgh 

I'licrto I'.'co 



ll 

ir 
n 
I 
I 



"75" 



70 
70 
70 
70 



16 
4 

6 



4 

T 

6 
6 
8 

6 
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sites have on-site comparison cfroups. Though 7 more Planned 
Variation sites have matching off -site comparisons, 17 
Planned Variation sites have no matched comparison groups 
at all. Fourth, inspection of Table II- 1 reveals that 
while the sites are generally spread around, the country, 
for some models there is little spread in site Ideation. 
For example, all of the Gordon sites are in the South, 

Each of these observations serves to complicate the 
analysis. Thus, while the structure of the design — 
sites nested within models and Planned Variation and 
comparison classes nested with sites — appears relatively 
straightforward, there are complications involved in 
carrying out a conventional analysis. 

Data Collection Activities * 

Colxunn 4 in Table II-l indicates the level of 
testing and evaluation carried out in the various sites. 
Primarily because of economic constraints, not all 
children in all sites were tested on the full range of 
measures. There were three levels of evaluation activities. 
Table II-2 describes the activities at each of the three 
levels. Level I is the most basic. Nine Planned 
Variation sites fit this category. Only one comparison 
site is in Level I. No c^ata gathering at this level 
involved the children. Teachers completed demographic 
information forms and filled out the California Social 

Er|c ^'^^^^-3 
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Table II-2 

Three Levels of Planned Variation Evaluation Activitits 

Level of Hvaluation Data Collection Period 

Level I Pall Spring 

1) . Teacher completed classroom information 

forms *•* for child demographic data X X 

2) Teacher completed California Social 
Competency Scale one for each 

child X X 

3) Sponsor ratings of Level of Implement- 
ation XXX 

4) Head Start Directors ratings of Level 

of Implementation XXX 

5) Teacher and Teacher Aide suTve>- • X 



Level II (includes all activities in Level I and the following) 

6) Classroom observations XX 

7) Basic Child Test Battery . * X X 

a. Preschool Inventory 

b. NiU Book 3D 

c. NYU Book 4A 

d. Motor Inhibition Test 

a) Child completed Ethnic Heritage Test X X 



Level HI (includes all activities in Levels I and 11 and th« following) 

9) Stanford-Binet testing on random one- 
half of children in all tested classes X X 

10) 8«Blocks Sort Task given to other 
random one-half of children in all 

tested classes X 

11) Parent Interviews administered to 
parents of children taking the 8-Block 

Sort Task X 

12) Tnunsivc Case- Studfes (I), of Maryland) x 
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Competency Scale for each child in their classrooms in 
both the fall and spring, and both teachers and teacher 
aides responded to a questionnaire requesting information 
about their own backgrounds, teaching experiences and 
attitudes. In addition, model Sponsors and Head Start 
Directors rated the level of implementatioA in the class-- 
rooms in each site. 

All data collected at Level I was also collected 
at Level II. In addition, three other sets of data were 
gathered at Level II. Classroom observations were made 
in both the fall and spring by observers using the SRI 
Classroom Observation Instrximent (see report on 
••Implementation in Planned Variation — 1970'-71")* All 
children in tested classrooms were administered the Basic 
Test Battery in both the fall and spring. Four tests 
were included in the Basic Battery — the Caldwell Preschool 
Inventory (PSI) , NYU Booklet 3D, NYU Booklet 4A and the 
Motor Inhibition Test. Finally, black and Spanish 
children whose parents were willing took a test assess-- 
ing their knowledge of their ethnic heritage. Ten 
Planned Variation sites were classified as Level II. 
Of the ten sites, four had on-site comparison classes which 
were also tested at Level II. Finally, two of the ten Planned 

Variation sites had off-site comparison classes tested 
at Level II. 



I 

i 

I * 

/ 

Level III bxtos bad all' the data collection carrj^e^l 
out in Levi&l I and L-avei IX .s:.-tes and* in addits^c^r;^ f©ur' - 
other activxtias wc^rs ^dd©4^^ ' One ramder^^ly clios*io Half 
of the children in each testc^d iM^ml 111 c\m%tQm^ ^v#r€t 
adndnistered tiie Stanford-Bxriat iiii bath the Fall ^Mid th#. ; 
Spring..,^;- the sam# chx-i^frSin received tl\% test both tiP-es* 
The children xn the otMer half of the class # aloBg ^ith one 
of .their parents or guaordianS/ v/iire admfiistered the 8- 
Block Sort Task in -'tii# Spring* AdditioMily# the parei^ts 
or guardians of the t^hi Idren in. this ,fr©up complected a 
parent questionnaire ^^^hxch asked about attitudes t©^*(^trd 
Head St^^rk, thexr -t^hild. and thm Planned ¥ariati©]^ inodel 
used in thair child'*s classrooin. Finaily^.j a spall rsmbi^r 
of children in eaeh of the Planned Variatiori L^vcil III sites: 
fonrted tha saijpia for an intansiv© easa sttudy ea-rried out 
and _ reported by the U?u varsity of Maryland Cscg# fla ad Start 
Planned Variation Case Studies 1970-711 « Eie|htaeju 
Planned Variation^, ten on^-sitc^ ^iivlnparison, ar^d fiv*i off-, 
site cojiparison sit^ifs were assigned to L-^vel lO, 

gescrigtions o £ _ Data ;C"ojj_5yti©n^_^ l5il£HS£Bl5 

"Th0 Oualxtv of Planned Variatxan Data*' ^.tecr^b^t^ m 
detail many of the :instru:rt^;?nts usad xn tfu^* i^tuay* Thc^ 
report on I mp 1 t a t a on xn Planned Variation I'S?©-?!"' 



^Bcrilt^n tn detail mo^t o£ the r^mt* We or-ga readers 
who want more than a cursory descri|.^tion to refer to tbose 
repo»-tik In _thi.s section^ we mmmly xndieatct the principal 
ir-i-v-ft ofNiht^ ins trumarrts J. briefly describe hm they were 
used in 'the c3a^i*-*^«tfi lectio and note wheth.er wa wiil be 
ming data from tiieim m this report* 

Before describing the instruim^nts, Botm n^entiors should 
be madm 6f th€ ^strate.gy for data colleotioE Bsed by th# 
Stanford lesearch Irmtitute* For questionnaires completed, by 
teachers teaeh^r aidei^ and Head Start directors^ the approach 
was to mqmBt that the forms bm filled out and to pay the 
resporjdarits a sitnall stipend for their tin^* If the forms 
vera isicoj^^plata or patently ^naccyrata 'they were rettirnad 
to^ the person ^ho filled them oiut with a reqiiast for clari- 
fication. In som# xnstaBces"^ as with thm Class roaiDi In for-- 
joation Forms filled out by the? teacher^ tins process was 
repeated a ms.nfcor of tiw^s. Generally *a site co^-ordinator 
was prfesc-nt to emcourage teachers and Head Start directors 
to fioi&h their forms quickly aiid accurately. 

The Bitm coordinator was also respoiisible for 
hiring, tr^min^, and oymrsminq a staff of local testers* 
■Site coordinators v^rcr local p^dpU initially trained at 
SBl headquarters m Menlo Park, Cal.-' The reason for 
using local testers was to imum that sufficiant 
rapport t^^iisted be>t^*^€5n teachers and children* 8v 
ami larq'cu SKI persor^nel visited every site during the 
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testing poriod to answer questions about procedures and to 
evaluate the quality of the tasting- The main exception to 
this general procedure was thM Stan£ord*-Binet testings. 
Hercf^ certified testers from as near the local sites as 
possible we ire 'hired to do the testing* 

ffha classroom observations were also carried our by 
locally hxred personnel after 'they had been extensively 
tr^iined by SRI personnels The Case Studies were completed 
by students «md faculty* from the University of Maryland. 

Testers and observars were instructed to fill out a 
short questionnaire after /they had tested each child^in- 
dicating problems with the test session* The responses 
to these questibnnaires have proved to be very useful in 
the data cleaning effort preceeding data analysis. When 
the data ware gathered from the sites they were returned 
to SRI and subjected to a careful screening before being 
placed on IBM cards and subsequently on magnetic tape* 

By and large, we have been very pleased with the quality 
of the data. For an evaluation of the data gathered in 
lt^71-72f the reader is referred to '^The Quality of the Data 
in Planned Variation 1969-72", 

The following briefly describes ^ach of the instruments 
Used in 19V0-71. We also indicate the extent of each .instru- 
ment's .use in this and other reports in this series. 
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1. Classroom Information Form : This instrument was used 
to gather information about the background and family 
characteristics of every child in the sample. Teachers 
completed the instrument by gathering information from 
Head Start application blanks and interviews with 
parents, A validity study of selected items from a similar 
form used in 19 71-72 yielded encouraging results (see "The 
Quality of the Data"). Information from this form is heavily 
used in this report and in the report "Cognitive Effects of 
Preschool Programs on Different Types of Children". 

2, California Preschool Social Competency Scale ; This is 
a teacher completed rating scale of 30 items designed to 
"measure- «he adequacy of preschool children's interpersonal 
behavior and the degree to which they assume social respon- 
sibility" (Levine et al., 1969, p. 3). An extensive des- 
cription of the measure is included in "The Quality of the 
Planned Variation Data". This measure is only briefly 
analyzed in this report. Completion of the scale by 
teachers suggested to us t that among classroom and among 
site comparisons would be illegitimate. The reason is 
simply that teachers may consider their own classrooms • 

as the reference group for rating students. Since the " 
compositions of classrooms vary greatly, the ratings may 
lose comparability when they are taken out of the immedi- 
ate ^context of their classroom. 
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^* Sp onsor Ratings of Implementation : This rating form 

is fully described and analyzed in the report on "Implementation" 

4. Head Start Directors Ratings of Implementation ; This form 
is similar to the Sponsor Rating except that it was completed 
by the Head Start Director. It is discussed in the report 

on "Implementation". 

5. Teacher and Teacher Aide Survey ; These forms assess 
teacher and teacher aide background, teaching experiences and 
attitudes towards the Planned Variations model. They are ^ 
extensively analyzed in the "Implementation" report:' In 

this report, we .use some items taken from these ' 
surveys. 

6* Classroom Observation Instrioment : This is a broad range 
objective observation instrument developed at the Stanford 
Research Institute to assess the degree of implementation 
of classroom processes and child outcomes in the various 
programs. Trouble with the coding on the classroom obser- 
vation tape limited our use of this important instrument. 
An analysis of some results, from it are included in the 



\0 



-27- 

report on "Implementation" and an extensive analysis of 
its use in 1971-72 is under preparation hy SRI. 

7. Basic Child Test Battery : Four tests are included in 
this batteiry. The results from these tests are extensively 
analyzed in this report. Additionally, results from one 
of the tests, the Caldwell Preschool Inventory, are used 
in the report on "Cognitive Effects of Preschool Programs on 
Different Types of Children". Complete descriptions^ of the tests 
are in "The Quality of the Planned Variation Data". The four tests are: 
a. Caldwell Preschool Inventory . (PSI) The PSI was 
developed to assess general achievement in pre- 
school in areas deemed necessary for later success 
in school. Specifically developed for- preschool 
populations, 6 4 items tap areas of general knowledge, 
listening and word meanings, listening and' comp- 
rehension, writing, copying, quantitative skills, 
and speaking and labeling. Though the test was 
originally designed to have four factors, factor 
analyses of our data revealed only one factor which 
seemed to cut across all areas tapped by the test. 
Consequently, we simply summed the items to create a 
score on the test. Internal (KR-12Q,) reliability is 
roughly .90. (See "The Quality of the Planned Varia- 
tion Data".) By and large, wfe consider this test 
a measure of general achievement in preschool. 
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The scoring procedure for the test is not normed 
for age and as a consequence, pre-scores on the 
PSI are highly and positively correlated with the 
age that the child enters the program. The PSI 
also correlates roughly 0.50 with the Stanford- 
Binet, which in turn has a sli^ghtly negative 
correlation with age. The Stanford-Bine t IQ 
score is obtained by dividing .a calculated Mental 
age by chronological age the division by age 
makes the IQ score comparable across ages. The 
Mental age score taken alone can be thought of as 
the Binet score uncorrected for age. Mental age 
on the Binet correlates roughly .75 with the PSI. 
Assuming both tests have a reliability of .9, we 
find that the correlation among the "true score" 
parts of the PSI and the Binet score \anadjusted 
for age is roughly .83*. Though this correlation 
is far from perfect/ it uggests that the two tests 
are tapping somewhat the lame domain. 

b, NYU Book 3t) . The NYU booklets were designed to measure 
areas of specific preschool achievement. Book 3D 

*The sample used for ihese estimates and other estimates on 
fol'ilpwing pages of this chapter was ^he same sample used for 
...the ^correlation matrix on Page 120 o^' "The' Quality of the 
Planned Variation Data" for estimates of .the reliabilities 
of the two tests. > 
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is diftsigned to tap achievement in pre-math (seven 
items), pre'-science (seven items), and linguistic 
concepts (five items assessing knowledge of pre- 
positions). Both NYU'booklets (3D and 4 A) were 
extracted by SRI from the Early Childhood Inventories 
developed by A. Collier and J. Victor at the Institute 
for Developmental Studies at the NYU School of Educa- 
tion. Two scoring systems are used in the analyses 
in this report. First, a simple summary score 
obtained by adding together all correctly answered 
items is used. A factor analysis of the Book 3D 
suggested that there was only one stable, interpretable 
factor.* Estimates of internal reliability for the 
total score are generally in the range of 0.60-0.70. 
In this report we use 0.65 as a reliability estimate 
' for individual scores. Moreover, the single score 
seems to have a ceiling problem for some groups of 
older children on the post-test results. See "The 
Quality of the Planned Variation Data" for discussions 
of these issues. Second, a set of scores is obtained 
by considering the three sub-^tests as criterion- 
referenced measures. Using these measures, we 
report the percentages of children' in various sites 

*A factoj- analysis of Books 3D and 4A together convinced, 
us to keep the tests separated for analytic purposes. 
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and models for each sub-test who obtain either 
a perfect score or only one item incorrect at post- 
test time. We also report the percentages of 
children in these groups who f^il to get more than/ 
one item correct on each sub-test. 

A score derived from a summing of correct 
items for Book 3D bears a very strong relationship 
to the PSI. By and large , different isub-samples of 
the data reveal correlations of about 0.70 at pre- 
'test time (see page 176 of "The Quality of the 
Planned Variation Data") . Adjvistment of this 
correlation for the reliabilities of the two tests 
(PSI reliability, is roughly 0.90 and Book 3D relia- 
bility is roughly 0.6 5^> yields a corrected correla- 
tion coefficient of roughly 0.95 indicating that the 
two tests are tapping almost entirely the same domains 

c. NYU Book 4A. This test is designed to tap achieve- 
ment in three areas: knowledge of alphabet names 
(nine items) ; knowledge of numeral names (six items) ; 
and knowledge of shape names (three items) • The 
development of scores for this test was similar to 
the development of scores for the Book . 3D** A 
single sumimary score is analyzed in this report 
along with three criterion- referenced measures. With 
the exception of the third sub-test we follow the 
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same rules for creating our criteria, as we did 
for Book 3D. In the third sub-test, we required 
that the student answer all three questions correct- 
ly « to meet the criterion. The single score on 
Book 4A h^s an internal reliability of 
roughly 0.65 for the pre-test. To some extent 
this reliability is reduced by a minor floor prob- 
lism in the Fall testing. For all groups the Book 4A 
scores were positively skewed in the Fall and more 
normally distributed in the Spring. Pre-test scores 
for Book 4A and the PSI correlate roughly 0.45-0.50, 
with the Book 3D the correlations are roughly 0.40- 
0.45 and with the Stanford-Binet , the correlations 
are roughly 0.40. Overall, then, though the Book 4A 
is assessing somewhat similar areas as thePSI, 
Book 3D and the Stanford-Binet , there is consiaerable residu- 
al unique variance associated with the test. 

Motor Inhibition Test . This test was developed by 
Hagen and Degerman (see Maccoby et al., 1965) to measure 
a child's ability to inhibit movement when the task 
demands it. Three tasks are used to assess inhi- 
bition, the Draw a Line slpwly task, the Walk slowly 
task, and the Pull Truck slowly task. Four pre- 
liminary items assess the child's understanding of 
the concepts of slow and fast:. A substantial propor- 



tion of the sample of children in this study 
(over 50%) failed to ansv/er two or more of the 
four pre-test items correctly , in either the Fall 
or Spring/ indicating that these children did not 
understand the two concepts. The scores on the 
Motor Inhibition test were not analyzed for these 
children. Analyses of the three sub-tests indicated 
that the first two tasks yielded scores that cor- 
related roughly 0.46. Correlations of the first 
two tasks with the third task were roughly 0.24. 
The low correlations with the third sub-task indi- 
cated to us that it was either unreliable or was 
measuring something other than the other two sub- 
tasks. Consequently, we formed a measure of the 
Motor Inhibition by summing the amounts of time in 
seconds taken to complete the first two sub-tasks. 
Following Maccoby's lead and an inspection of the 
data, the log of this score was then taken. The log 
transformation removed the strong positive skewness 
from the new scores. This final score correlates 
in the 0.30 to 0.40 range with the NYU 3D and PSI 
and in the 0.15-0.20 range with the Book 4A and 
the Stanford-Binet . 
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8. Ethnic Heritage Test : Two tests were actually used 
here. The Ethnic Identity Questionnaire (EIQ) was developed 
by Manuel Ramirez III at the University of California, 
Riverside, to investigate the ethnic identity of Mexican- 
American children and the Children's Cultural Awareness 
Scale (CCAS) v;as developed by Edward J. Barnes at the 
University of Pittsburgh to explore the cultural awareness 
of Black children in the Head Start Planned Variation Study. 
Scores from neither test are used in this report. 

9. Stanford-Binet ; The Stanf ord-Binet Intelligence 
Scale is a well-known measure of "general intelligence". 
The 1960 revision was used 'in this study. A single 
measure of IQ is used ii this report. After extensive 
checking for matched pre- and post- birthdates and valid 
items, the score was calculated by dividing a child's 
Mental Age derived from the test by his chronological age 
in months and then corrected for age-related fluctuations 
in variance using the revised Pinneau tables (see Terman 
and Merrill, 1960) . 

10. 8-Block Sort Task Test ; The Eight Block Sort Task 
is a measure of maternal teaching style and interaction 
styles betv/een mother and child. ,The score used in this 
report ranges from 0-8 points and indicates the success 
of the mother in teaching the sorting tasks to the child. 
(See "The Quality of the Planned Variations Data" for an 



T-rki^' extensive discussion of this measure.) 
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11. Parent Interviews : this measure assesses parents' 
attitudes toward their children, Head Start and Planned 
Variations curricula. Although the interviews were con- 
ducted with only a small number of children, some of the . 
items in the interview are analyzed in this report. 

12. Intensive Case Studies ; In all three years of 
Planned Variations, students and staff of the University 
of Maryland's Institute for Child Study did extensive case 
studies of a few selected children in Planned Variations. 
(See Dittman and Kyle, in press, for 'a report of these 
efforts.) 

The Curriculum Approaches (Models ) 

This section briefly describes the twelve models used 
in the Planned Variations study in 1970-71. As we noted 
earlier, each of- the approaches, with the exception of the 
Enabler model, has been developed and is sponsored by 
some group of people in a University or private corpora- 
tion. The descriptions are intended to reflect the goals 
and expectations of the sponsors rather than to be. a 
^ critical analysis. As presented, they are idealized 
descriptions of the Ijwelve treatments. These sponsored 
approaches were included in Head Start Planned Variations 
because they were considered to be promising methods for 
working with disadvantaged children and families and be- 
cause they were unique in some significant way. Neverthe- 
less, the sponsors share common orientations. All of them 




sock to develop children's learning abilities* All are 
convinced of the importance of individual and small group 
instruction and frequent interchange between children and 
concerned adults. All attempt to make learning interest-- 
ing and relevant to the child *s cultural background. All 
believe that the child's success in learning is inseparable 
from his self-esteem, motivation, autonomy, and environ- 
mental support, and all attempt to promote successful 
development in these domains while fostering academic goals* 
The sponsors differ among themselves chiefly in the priorities 
which they assign to these objectives and in the sequences 
through which they pursue them. 

It is important to recognize that the concept of 
Planned Variation was not intended as a means of finding 
a single "best" method for educating disadvantaged children* 
A wide variety of groups of children are included in this 
study, and a program that is appropriate for some may not 
be appropriate for others. Some approaches, for example^ 
are primarily concerned with parental involvement and 
community control, while others place primary emphasis 
on the curriculum, the teacher / and the classroom* The 
following paragraphs briefly attempt to capture the emphasis 
of each model. 



Kduca^t fona llMfveloptv^^* t " Corpora t ion CKDC! 
Sponsor^ Contact; Scurge Hc^n 
• KDC .has an opon el/issro^^r^ approach d^^riwd f rem the 

British primary schc^^i i^ode^ t^uc^ari'i^s of chUJ d^v^^lon- 

inent* It belxeve:^ that Icfarni^rr ^ f.ici.U touted by iictiw? 
participation in tho rvTece*^s» Tnt:^ cl-u'-ror-^f? fj^ev^dt^r^ a 
setting in which ^there is a ■^ran^gc of i^vat«:^rxals and ^^icixvities 
from which the child can chcosc M^admmc skills are^ 
dev^aopoci in a self ''directt?d t^ay ti^rough classrooiB ^5cperi« 
onces* ■ The rola th^'* tevici'^er one ©f . leading the child 
to extend his own v;ork and generally xn%tolv^s vorKxng vith 
an individual child or smll ^roap* 

The Systeiratie Use of, 

University of Oregon ' ^ ■ 

Sponsor Contaet ; ^qbI<qy Becker 

The prijnary focus of the EngelPann-Bc^cker prograp 
xs on propoting _f>kills and concepts essentiai to r^^adipg^ 
arithit^et ic and lat^qnuq^ achieverient ^ %i?xth particular 
emt^iauiB on rowdy ^ng language dc^f ici^neii^s* The tii^m 
t«:»chnxques are programmed ^^at^^rxalfi, stracfeyred r^j>id=- 
fxro drills, and positive remf orc^r^^nts of rewards and 
praise to oneourag^ desired p?atterns ©f behavior = SimII 
study grotsps of fiV^-: to ton children arc arq<mi^eu &/ 
toachors aecordm^l to ability levels in orde^r tP faeiUtat?- 
^ present^^tiou of t^attt}rnc^d l^^arnim r^^^^ater iais and to ©lieit 
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Tlie Barak Streot ColU-mj oi ■Mnaatxon Approach 

St^Dfir.or^ Contact t Elx2;abeth GiJLkesorit • 

The Ba*4k Str^at approach ai^phasissos both learning .and 
-Botyial^^motx^n^l aevelopm^'Ht of children on tlm, ptQniB^j 
that they aru mtertv^m^d. The* teacher funeti.oiiis as a 
supportive ;^diil.t ^^hm^ the child can tryst ..^ and teaches by. 
relating and i^npandmq upon each cruld^s resp©ns€- to his 
€mpim€.nc€r^> ThQ clas.£.r©op is vie^Mtd as. a stable environ- ^ 
j^erat arid vcsrkrooin for th^ child 11=1 i^^hich he? is encouraged 
to QHplot^^ 5rake CHOICES and carry out plans. Academic 
skills are pr^^sc^ntea in the content of classroom oxperieoces. 

Siif^sort^aiia'' Cofitdr for Follo^^-Throngh^ Oniiversity 
of Kasisas 

Tfucr Bc^havic^r Aa^iiiy^^ita ar^^ roach has three? predominant 
a^^pc^etL^ F;r^it it ei^:phatu^;v4 j^:.4drinic and soeial skills* 
Individualx^red pro^u^^irr^cd r^^t^-riain jre pr iwery 
tcf-ischingi ^c^^ecii:^ - -t' fttai^r-x svntci^at 44:; use ©f positive' 

nnnft^^^^'^.^rtK^tit A ^^okfcr^: Cf^Kcnanq^.^ "^^^y'^tc^i^ i% n^ed to suppc^rt 
chi Itlrri-n' k:.«riruny v;f fcrt tu^r-S it employs parc;ntv> as 
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Indt vidual ly Proscribed Instruction and the Primary Edu cation 
L€»ar rung "Research and ; <avelopment Center, Univ. of Pittsburgh 
Spons>or Contact; Lauren Resnick' 

rhe IPI approach provides an ' individualized program 
of instruction for each. child which teaches him academic \ 
sk-ills and concepts in the areas of language, perceptual \ 
.ntotor mastei-y/ classification, and reasoning. The materials 
arer sequenced to reflect the natural order in which children 
acquire key skills and concepts.' Diagnostic tests determine 
each ohild''s strengths and weaknesses and are used by the 
teacher to prescribe- instructional materials appropriate 
to his needs. Positive reinforcement, both social and 
concrete, is given continually for success in learning. 

The Respons ive Environments Corporation Model (REC ) 
Responsive Envi rontnen ti~ Corpora 1 1 on ^ ~~ ' 

Sponsor Con tact; Lorj, Caudle 

The REC rood€?l uses specially designed,, .self-correcting 

multi-sensory learning materials which strengthen school 

readiness skills in language and reading. They are designed 

to teach basic concepts while allowing children to make 

choices, vmxk independently ,^^^^1 set goals for themselves. 

Teaching pachmet- in the form of "talking typewriters" > 

and "talkarKi pages" involve children 'in learning, by seeing, 

traci,n%%. typing/ ^nutating and discriminating among sights 

and soundfi and by record msj and listening to titeir own 
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% The Flo rida Parent Educator Model 
University of Florida 

Sponsor Contact : Ira Gordon 

The Florida approach is not /"a specif ic classroom 
instructional model but is designed to work directly in the 
home. It focuses on the parent, believing that the parent 
is the key agent in a child's development. The majpr goals 
of the program are to^d^eVelop educational competence in 
the child and to develop an atmosphere in the home which 
will foster continued growth. An important role is played 
by paraprof essionals called parent educators. The pafei)t 
educator spends half-time with the teacher in the class- 
room and the other half making home visits. The home visit 
involves bringing tasks into the home and instructing the 
mother how to teach them to the child. 

The Tucson Early Education Model 
University of Arizona 

i Sponsor Contact : Ron Henderson 

( The Tucson model has a flexible' child-oriented 

curriculum which focuses simultaneously on four areas of 
development: language competency, intellectual skills, 
motivational skills and societal skills. Emphasis is 
placed more on learning to learn skills than on specific 
content. The content is individually determined by a child's 
environment and interests. The classroom is arranged in 
interest centers for small groups. The teacher's role is 
to work on a one-to-one basis v/ith the child, arrange the 
Q classroom setting and encourage interaction^ between the 

ERIC 
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Responsiv.e Educational Program ' 
Far West Laboratory for Educational Research and Development 

Sponsor Contact ; Glen Nimnicht 

The Responsive Educational model emphasizcc celf- 

rewarding learning activities and a structured environment 

responsive to a child's needs an^ interests. The model 

encourages the child to make in1:errel^ted discoveries 

about his social world and p^hysical environment and 

i 

Stresses the importance of^the development qf a healthy 
self -concept. The classroom is a controlled environment ^ 
in which the child is free to explore various learning 
centers, games and activities. Problem solving and concept 
formation as well as sensory and perceptual acuity are 
stressed and the pace of ail learning activities is set 
by the child for himself. 

Cognitively Oriented Curriculum 
Hi/Scope Educational Foundation 

Sponsor Contact ; David Weikart 

The Cognitively Oriented Curriculum combines Piagetian 

theory and an open classroom approach. It uses a cognitively 

oriented curriculum and emphasizes the process of learning 

rather than particular subject matter. It stresses a 

child's active involvement in learning activities. The 

teacher takes an active role. Additionally, home training 

is seerr-a^.^?a>t of the program and the teacher suggests 

tasks for the\ mother to present to the child at home. 

■ ...'1 
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The Enabler Model 

Office of Child Development 

Sponsor Contact ; Jenny Klein 

The Enabler Model does not involve affiliation with 

a particular instructional approach. It is build on goals 

prescribed by each community for itself. The development 

and implementation of this model are facilitated by the 

assistance of an OCD consultant who takes a very active 

role in all aspects of the program. Thus projects with 

the Enabler Model may differ considerably in the approach 

and style of their educational tactics, but all share 

a commitment to high levels of parent participation in 

policy making, program planning and classroom operation. 

The' Independent Learner Model N 
New York University 

Sponsor Contact ; , Don Wolfe 

In the Independent Learner model, learning occurs 
principally in structured small-group instructional "games" 
where children of different ability levels teach one 
another and become relatively independent of the teacher. 
The verbal interactions among children are implicit in 
the process and are a direct stimulus to language develop- 
ment. Experiences in phonics blending and decoding skills 
stimulate reading ability and language~math-~logic games 
such as Cuisenaire rods and matrix boards promote mathematical 
comprehension. 
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chapter iii 

charactehjstics of children, classrooms anp-^sites 
in the 1^70-71 planned variations sa^iple 



Introduction: 



This chapter has six sections. The first section de- 
scribes the 37 Planned Variation sitjes . After criteria for the . 
selection of sites are discussed^ characteristics of the child- 
ren in the sites are summajrized, for each site.- Additionally, 
data on location and structural characteristics of the sites , 
are shovm. The second section describes the comparison sites 
in the same way. Section three describes the strategy used to 
reduce the total sample to a v;orking analysis sample. Section 
four describes the analysis sample by child characteristics and 
classroom characteristics. Section five contrasts the Planned 
Variation and Comparison analysis samples. Section six reports 
analyses comparing pre-scores in the Planned Variation and 
Comparison samples. 

I The Thirty-Seven Site Planned Variations Sample 

The selection of the thirty-seven Planned Variation sites 
had a large part in determining the overall design of the study. 
Although we briefly mentioned the distribution of sites by 
models and testing level in the second chapter, it is useful to 
give an overview of this information before we discuss the 
criteria used to select the sites. 

Er|c 'OS 4 15 
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Table III-l shows the number of sites for each of the 
twelve models in the study. The models are crosi-claiiif ied 
by the level of testing used in the study. Four thingt ihould 
be noted in Table Ill-l. Firtt, for analytii purpoMi th«r« 
arc really only 11 models. The Virgin Islands if the only 
NYU site and it is a Level I. Second, tt^ other models (Pitts- 
burgh and REC) have only one site. Although we carry out 
extensive analyses of the outcomes of the programs in these 
sites, inferences about the effects of the models are weakened 
by the fact that there is no experimental replication. Third, 
there is no Level III Enabler site. Thus, we have no Stanford- 
Binet or 8-Block Sort data for this model. Fourth/ for the 
remaining eight models an attempt was made to have at least 
two Level III sites and one Level II site per model. 

TABLE III-l 

Distribution of Planned Variation Sites 
Within Models by Testing Level 



Mode 1b 



Totals 





Far West 


Tucson 


1 Bank St. 


Oregon 


Kansas 


Hi-Scope 


Gordon 


EDC 


IPI 


REC. 




Enablers 


Total 


I 


2 


1 


1 






1 


1 








1 


2 


9 


II 


1 




1 


1 


1 


1 


1 


1 








3 


10 


III 


2 


2 


2 


2 


2 


2 


2 


2 


1 


1 






16 




5 


3 


4 


3 


3 


4 


4 


3 


1 


1 


1 


5 
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In Chapter I we described three criteria that Planned 
Variation sites had to meet in order to be included in the 
study. Briefly, each site had to have, existing Head Start 
classes r each site had to be located in an area that fed into 
the Fellow-Through school or schools, and each had to adopt 
the curriculum model used in the Follow-Through school. With 
the exception of the Enabler model sites, the same criteria 
applied in 1970-71 as in 1969-70. Other criteria, however, i 
were also used in 1970-71. The expansion from 16 to 37 sites 
reflected a variety of design and political constraints. 

In addition to the three previous constraints, the 
choice was influenced by an attempt to have three or more 
sites for each of the original eight models, by an attempt to 
expand the number of models from the original eight, and for 
reasons of geographic representation. A final constraint was 
imposed by the budget of the study. One result of these often 
conflicting constraints was that the characteristics of sites 
within models differed from model to model. Table III-2 dis- 
plays the 37 sites grouped by model and contains some summary 
structural and demographic information about them. As nct0d 
earlier, at least demographic and the California test inform- 
ation were gathered in each of these sites. 

A few things are clear from Table I1I-2. First, there is 
wide geographic diversity. All regions of the nation are rep- 
resented. Second, large Northern cities are clearly unrepre-* 
sented. There are no sites, for example, in New York City, 
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TABLB III-2 



Characteristics of Head Start Planned Variation Sites 
Total Sample 1970-71 





Testing 
Level 


Region* 
Location 


a 

U4 U 
O 'O 


Entry Lfvel 


% Previous 
Preschool 


c 

•H 

II 


% Black 


66 


■ 

< 
1 

K 

ji 


M 

i. 


o 

£ 


S")NHOH 


CODIi 


SITH 
























NTmn4 fhf 
































T 
1 


NU 


178 


K 






75.7 




"■J76 


6 6 


X 1 » tJ 


02.04" 


Duluth 


tlx 


mi 


l46 


V 

t\ 


i i . / 




— JTT* 

D .9 


a'a 

V . W 








02. or 


Fresno 


ill 


Kill 




If 


"7 0 


TTTT 


59 . / 




r4.3 


' 676 


0.0 


02.09' 


Salt Lake 


T 


Kill 








"SO" 


X 1 .u 


0.0 


'22.6' 


0.7 


64 .0 


02.13" 


Tacoma 


II 


Nil 


138 


V 

h 


. *i 




91 . ^ 


0.0 


1.4 


T.2 


58 .,7 




























TUcson 




























05.08 


LaFayette 


in 


SR 


399 


1 

X 




59*2 




6". 6 ^ 


"6 0 


6."0 


69 .8 


03.09 


Lakcwood 


X 


NU 


"125 


V 


u . o 






25.6 


0.0 


0.0 


15.2 


03.16 


Lincoln 


TTT 


NU 


170 


V , 


4.2 


55.0^ 


ft/ 




5.3 


411 


62.8 




























B ik 
Sweet 




























05.01 


Boulder 


TTT 


Nil 


66 


V 

h 


A' Q 






0.0 


77^9 


3 0 


56.1 


os.Ttr 


'TusVegee 


"XI 


' "SR 


I6i 


* 




"SOT 


' Ow . 9 




"ft. 6 




"16. "T" 


0?.ll 


Wj Imlngton 


"TT 


ou 


"ITT" 


If 


*! 7 ft 

i £ ■ V 




Al i 

VO . I 


0.6 


0.0 




— 173 


■ffS'.lF 


Blinira 


TTT 
111 


Nil 


li6 


Y 


OO . o 


UTT 


^d X 


"O" 


0.0 


0.0 


59.3 




























B :ker ^ 

E^jle- 

mann 




























07.03 




rifi" 

I X t 




"237 ' 


N 


'A A 

u . u 






"O" 


'T.6 


0.0 


0.6 


07.11" 


i 1> V X V/ 


L X X 


QII 


105 


1 
1 




T4T" 


0/ . 9 




■"3". 6 


0.0 


31.7 


07. IT" 


E. Las Veffas. Niti 






"114 


1 


A A 


'srr 


— A A 


6.6 


90.4 


0.0 


9.6 




























Bvshell 




























6S.62 


Oraibl 


TTT 

IX I 




$3 


v' 
N 




' 56' O" 


A A 
V . U 


A A 

V . V 


A' A" 






05.04 ' 


PoTl" .i(yi*vi 111* 




MR 


' 74 


N 






iA E 


TTTT 


6.6 


6.6 


59.5 


bi.SS ' 


Mnund ^ T 1 1 ^ 


TT 


NR 


^9 


t' 


9 . o 




M 7 


6.6 


0.0 


0.0 


36.3 




























W ikart 




























0^.02 


Ft. Walton Bch 


in 


SU 


90 


K 


0.0 


53.6 


66.9 


0 (T 


0.0 




31 . 1 


0^.04 


Central Oz 


I 


SR 


282 


K,l 


21.7 


60.1 


0.0 


10. 5"' 


0.0 


0.0 


99 . 6 


6^^.06 


Greeley 


in 


U\) 


/7 


K 


16.9 




1.3 


5.2 


76.6 


6.0 


16.9 


O^.lO 


Seattle 


II 


NU 


104 


K 


lS.5 




49.5 


0.0 


6.0 


S'.6 


35.6 




























Gordon 
















^?1."7^ 












10.01 


Jacksonville 


1 


SO " 


60 


k 


13.3 




93.3 


0.0 


T.O 


0.0 


6.7 


10.02 


Joncsboro 


III 


SU 


68 


1 


2.9 




36.9 


0.6 


0.0 


ff.'O 


69.1 


lii.'O? 


Chattanooga 


LJl 


SU 


170 




1.8 


61.6 


83.9 


6.6 


6.0 


6".o 


16.1 


10.10 


(louston 


tl 


SU 


7J 


K 


0.0" 




63.5 


o.-D" 


35.1 


"irr 


1.4 ' 




























iXit 




























] 1 . OS 


Wash in;,' ton 


rii 






K 


17.6 


46.3 


69.3 


2.4 


1.2 


0.0 


4.8 


11.06 


Piiterson 


II 


NU 


135 


K 


0.8 


53.4" 


92.5 


6.0; 


0.0 




l.S 


11.08 


Johnston Co. 


rii 


SR 


117 


1 


32.5 


66.4 


53.0 


0.0 


0.0 


0.0 


47.0 














1 
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TABLE in-2 
(contM) 





Testing 
Level 


Region* 

Location 


No. of 
Children 


Entry Level 


\ Previous 
Preschobl 


Mean Age in 
Months 


% Black 


M 

*\ 


i 

t 

M 


S 

0 


% White 


S JNSOK 




SITH 


















\ 






Pitts- 
Jb Tiih 




























12.03 


Lock Haven 


iir " 


NU 


li6 


K 


is.6 






U.U 


-Vr 


'A A" 
-U.U 




















































— V- 






26.61 


Kansas City 


ni 


NU 


in 


k 


7.4 


S4.3 




A A 
U.O 


.4^*4 


A «i 
U.O 


Ji.d 
























V— 
































U.oi 


St. Thomas. VI 




-R 




k 


11. i 


TTTS 


92.0 


4.3 




-k<r 


3.7 
























=^ 




E^ablers 




























27.04 


Billings 


II 


NU 




1 


9.8 






"6"nr 






64 .6 


27.05 


Colorado Sp 


n 


NU 


ibS 


K 


•6.S 


55.8 


TTT 


6"nr 


■4r,>- 


■67s 


24.1 


i>.6i 


Bellows Falls 


ii 


NR 


i03 


K,l 


i6.r 


55.6 


0.6 


nnr 


-5.6 


-o- 


100.0 


27.02 


Newburgh 


I 


NU 


Hi 


k 


0.1 


52.5 


TO" 




-076 




15.6 


27.01 


Puerto Rico 


I 


-U 


104 


K 


4.8 


64.7 


6.0 


ibo.o 


0.0 




0.0 




























- TOTAL 


4974 




14.2. 


56.6 


47.0 


3«6 


8.2 


2.7 


37.9 



• Tho first code is either N or S, ittnding for North or the old South; 
the second code is either U or R, standing for either Urban or Rural. 
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\ 



piiiladelphia, Chicago, San Francis'co or Detroit. The cities 
^t^presenting the l^orth — e.g., Buffalo, Salt Iiake, Elmira, 



^ Wilmington, and others — should probably not be thought of . 

as similar to the large metropolises. 

— Th^rd, sites and models differ in a<srs^r#gat# characteris* 
tics of, children. Some sites are predominai\tly Black while 
others are predominantly white or Mexican American. In some 

- ^ sites most of the children have had preschool experience — 

in others none have. This hplds for models as well; in the 

— . Florida ircdel, for example, very few of the children had prior 

preschool experience while in other models a considerable percent* 
age of children had previously attended preschool,. By and large, 
children in the Soiith are older and the probability of a child 
having had previous preschool experience is slightly greater if 

— he is in the South. A number of Southern sites* and one 
Northern site (Billings) ^end their children directly into first 
grade from preschool — we call th^se Entering First sites (El 
sites). Chattanooga, Central Ozarks and Bellows Falls graduate 
children into both Entering First and Entering Kindergarten 

— claisses. The majority of Chzittanooga children enter El 
classes and the site is classified as El for analysis purposes. 
Central Ozarks is a Level I site and is not included in the 

\ 

\ analysis (see next section) . The majority of children from 

'\ Bellows Falls enter EK classes and the site ii clatsified EK 

V for analysis purposes. Variation in age among the sites is 

very highly correlated with elementary grade entering level 
(El/EK) . Older children attend El sites and younger children 
attend EK sites* At the classroom level the .correlation between 

\ 



*LaFayette, Tuskegee, Tupelo, E.\Las Vegas, Jonesboro, and 
Johnston County. 



i ; 1 
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mean class age and an El/EK site level dichotomous variable 
i^s 0.922*. The reason for this high correlation is clear — 
by and large, the Head Start experience directly precedes- ele- 
mentary school and most first grades enroll children at age 
six while most kindergartens enroll children at age five. 

We have focused on the variation among sites in 
ethnic background, preschool experience and El/EK because these 
variables are used extensively for control and stratifying pur- 
poses in the analyses described in later chapters. Each of the 
variables has a powerful relation to test scores and test score 
gains. , Analyses contained in the report "Cognitive Effects of 
Preschool Programs on Different Types of Children" suggest that 
there may be important model interactions with both preschool 
experience and ethnic background. 

Finally, there are both logical and empirical reasons for 
distinguishing El sites from EK sites in the analysis. Since 
the El sites are primarily Southern and involve older children, 
entering level might be viewed as .a proxy for region and age. ' 
Moreover, it is not difficult to imagine that preschool teachers 
in El sites go about their jjobs /somewhat differently than do 
preschool teaphers in EK sites. The fact that children iji.El 
sites will directly enter first grade might make the teachers 
conscious of a responsibility to prepare their children for 
beginning reading and arithmetic. Teachers in EK sites might 



* The sample used here is the analysis sample of 166 classes 
described in a later section of this chapter. 



well end up relying on kindergarten to share g^art of that 
responsibility, 

II Df.s cription of the Comparison Sample 

Table III--3 describes the comparison sample sites — in 
the same terms as the PV sample in Table III-2» There 
are 22 comparison sites described in this table* All but one 
(St. Thomas) are either Level II or Level III tested sites. 

* The remarks in the preceding section about the diversity 
among Planned Variation sites in ethnic composition / propor- 
tion of children who have previously attended preschool^ and 
age apply equally to the comparison sites. Although* there is 
great diversity among the sites, however, a brief comparison 
of Tables III -2 and III-3 suggests that there is considerable 
similarity within locations between Planned Variation class-* 
rooms and comparison classrooms. The similarity between mean 
ages of the paired Planned Variation and comparison sites is 
particularly great, and, while the pairing by sites does not 
always eliminate differences in racial composition and pre- 
school experience, it clearly has some effect on these variables* 

The selection of comparison sites deserves some discussion. 
By and large, there was an attempt to obtain a comparison site 
for each of the Planned Variation sites. The idea was to find 
Head Start classes not funded by Planned Variation in a nearby 
location _^or each PV site. In theory, the comparison classes 
could exist in the same centers as the PV classes, though in 
practice this did not occur in 1970--71. When a reasonably 
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e©f!fiparabie s.^-t 'f^-t Heaa Ss.art v^orc? found ir» the Planned 

Variat4©f5 ccr!:r":uf;sft;y , t© feu;? ef the classes were chosen t© 
have tii& tests unci rta^yras «siiri a i red t^^ tiief^. When it w^s • 
aff'pessifcl© iifv'i :i e=i*;t^p«r*sM^f s&e ©f Head Start classes trfith- 
isi! a c-c^f^afvity fa-^is, attef^pt t© f© ©tatiside ©£ th© com" 

mvntty te rsfcarJi--- Iseatis-ns, m fin<? cemparison. chiWr&fi. In this 

the aiiiaiysa? Si^ tihii tisc?;iir i&^lci, no cofsparaseh classes 
^sf-^jr*" Rfciie^t**;; . T-rr Sit'cftrf.t ft.:ii|6 tc- fiml eoRparisen -classes 

co??^«Rs.t i:; en aaaufrpt xGfi that the csr^e^risfeR 

cSd^sf^- -fefin t^: ?'i?i-L.r *?5f: rv classes in bH resp«icts eK- 
-^rrpt thfr-e ;f.- :: u^i- 1 -Tut ¥ TMs mmt^pt^mi 

&iir|-ch*%«-':: . '.f^f -j: i si ^i^^-E^iS-s Kay n*sve tnf^ueneed 

8?y tf»- art'. T'V 4,f..r:».yf;ity afia ftRas t^-Ct'Ct -s riii« 




cast?, whatt'ver biases existed in the initial selection of 
Follow Through schools were communicated to the PV .classes. 
In other instances, tho Head Start and CAP personnel could 
■ choose from ar^ona a variety of Head Start centers all within 
Follofe.' Throuv^h attendance zones. Since no clear guidelines 
existed to dcfterisirse thoir choice of PV classes, their selec- 
tion could have :^ntrodaced bias into the study. Speculation 
in this area is very difficult* Since we have no direct way 
ef wadrjrotandinn what biases exist in the sample, our strategy 
is to contr©: for as r^ueh as possible and hope that we are 
directly control lin-j the biases or that our control variables 
ir© str©n«3ly associated with the bias. 

The second proble.^ with the argument that Che coiaparison 
class-.^jR fnay not bo different fro.r. the PV classes except with 
rcsfM-rt c!i,f tcrencei;. crGated by the PV curriculum arises 
frc--- ^hi^ «>fi,;r-ctn of contaira nation. This issue only arises in 
p^act-^ wh«.'r!s,' oc#th r.e:kri and coisparison classes are tested. 
Briefly, it stejn.s frcr. the fact that facilities and consulting 
services availalsl© to &nly a so loot group of classes within a 
corsnaniey and net to ether classes fl»ay create a situation that 
1% ir.tGle-i-alslf rerally and pcli txcally for Head Start directors 
arid '©the r E-;it>f;rvisory p^rsonmi. In the case of the Planned 
.Variatiffi f,tuay, tht; P'J i-lafcsrosrss were rfeceiving extra equip- 
ju?,n* dn4 the- t c »»f yerc- receiving extra in-service and 

ps-.>"5,»>rvic»:' triiir.i.nt: bc-yond that available with the normal level 
of Ito^sd ^tisr^i :u:-.dln^^ in tho sctfi^tianity. In^these circumstances, 
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lt v7ouia not be unnatural for a Head Start director to let some 
of the equipniert intended for PV model classes make its way 
into comparison classes. -An,d it is natural for Head Start di- 
rectors to let comparison teachers attend some pre-service and 
tn-service training sessions. This situation might be aggra- 
vated in a community where the Head Start director was enamoured 
of the particular model being used in the model classes and not 
particularly impressed by the importance of the evaluation. 
Over the course of the year 1970-71, reports from the OCD con- 
sultants indicated that some contamination was occurring. When 
this evidence was known to SRI and OCD before sample selection 
for 1970-71, care was taken to exclude heavily contaminated 
classes from the comparison sample. 

It is, however, practically impossible to estimate even 
roughly the effect that the contamination had in the various 
communities so, by and large, we ignore the problem in this 
report. If this type of study is to be done again, some sys- 
tematic way of estimating the influence of contamination should 
be devised. 

Ill Goneration of the Analysis Sample 

One problem faced in all analyses of large data bases is 
the creation of the sample used for analysis purposes. For a 
variety of reasons, all data is not usable in all analyses. 
Throughout this report, we will focus on one particular sample 
of 2,235 children, 'Jhe reduction from the original sample of 

O Hi. 1. 1 7 
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6,297 rostered children to the sample of 2,235 children had' 
two main steps. Fitst, we eliminated certain entire sites from 
the total sample and also eliminated all children in classes 
which were not tested in the evaluation. Second, we eliminated 
some children in the remaining sites and classes because of 
missing data. 

The focus for analysis in this report is on objective 
measures of the effects of different pre-school experiences on 
children. The major measures assessing these effects are the 
four cognitive tests and the Motor Inhibition test. The Cali- 
fornia measure, as noted in The Quality of the Data , should be 
viewed as a subjective child assessment. As such, it presum- 
ably has within-classroom validity but lacks across classroom 
validity. Since the children in Level I tested sites were not- 
administered any of the four cognitive tests or the Motor Inhi- 
bition test, there was no reason to include them in an analysis 
prepared to assess the effects of the cognitive tests. The 
elimination of the Level I sites reduced the sample from 6,397 
children to 4,864 students. Another reduction in size resulted 
from the elimination of two PV sites and one comparison site. 
Specifically, we eliminated from the general analysis sample 
both Oraibi and Fresno. The reason for eliminating Oraibi and 
its comparison site Acoma V7as simply that we felt they were 
not comparable either with other sites or with each, other. 
Both Oraibi and Acoma are American Indian reservations in the 

Southwest United States. We felt, for reasons of different 
O i <] .) ^ 

ER^ 



languages, cultures , and exp<?riences that neither site was com- 
parable to the other sites in the analysis sample, and we felt 
' ffi»r r^soj}JS,.©fv^ifJm;OJ% and cultures that the two 

'.'mm^ Wmm^^^^ to ' feach other . 

The second site eliminated from the analysis was Fresno. 
Fresno under\\rent considerable controversy during the school 
year over its Planned Variation model and at year's end, 
decided not to continue the model in the third year of study. 
This controversy not only seemed to affect the nature of the 
pre-school program as reported by the OCD consultant, but also 
influenced the quality of the data collection. After d6liber- 
ation with SRI personnel responsible for the data collection, 
iv;e decided that too many unknown biases existed in the Fresno 
data to make it a legitimate candidate for inclusion in the 
general analysis. There were no comparison classes in Fresno. 

Both of these sites were excluded on the basis of intui- 
tion and subjective analysis rather than empirical data, thus 
there is room for argument on the validity of the decisions. 

The elimination of these twp sites and the comparison 
classes in Acoma reduced the sample from 4,684 children to 
4,650 students. 

The next step was to eliminate the non-tested classrooms 
from the analysis. This reduced the sample to 3,131 children. 
After this step, the number of classrooms was not reduced, 
though the number of children was reduced. We then eliminated 
all children who did not have a valid pre-test or post-test 
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score on at least one test in the Basic Battery, One result 
of this was to eliminate all children who either left or 
entered the classes during the year. Thus a child was retained 
at this stage if he had a valid Fall: or Spring score for either 
the PSI, Book 3D, Bcok 4A or the Mjotor Inhibition Test. A 
valid score was determined by the tester./ 

The next step was to eliminate all children who did not 
have a valid pre-test and post-test score on at least one of 
the following measures: PSI, Book 3D, Book 4Ar Motor Inhibi- 
tion/ California, Stanford-Binet . Three final steps were 
taken. First, we eliminated all children who did not have a 
legitimate code for the background variables sex, age, pre- 
school experience and race. These variables were ne^cessary as 
key stratifying variables and would be difficult either to 
treat as missing values or to impute scores to. Second, we , 
eliminated all children with an ethnic or racial origin other 
than Black, white or Mexican-American. Specifically, we re- 
moved Puerto Ricans, American Indians, Orientals and, other non- 
Caucasian children from the sample. Our reasoning was that 
there were too few children in these groups for which to make 
reasonable comparisons. There were a total of only 47 American 
Indians, 31 Puerto Ricans, and less than 10 Orientals and other 
non-Caucasians in the sites included in the analysis sample. 
Third, we eliminated 22 children whose ages were under 44 months 
or over 74 months. These children were seen as distinct out- 
lyers and not at all representative of the rest of the sample. 
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This concluded our sample reduction and left us with 2,235 
children. 

It is reasonable to expect that had other analysts been 
responsible for the analysis, they would have developed dif- 
ferent decision rules. Our justification for those we deve- 
loped was that they seemed at the time to be reasonable. One 
indication that our sample reduction was not extreme comes 
from the fact that only 2,567 children received the basic bat- 
tery (in our selected sites) in the fall of 1970, Of these 
children, we retained 2,235 children, or 87%, Thus only 13% 
of the possible candidates for^ inclusion based orj Fall tests 
alone were eliminated for one of the following reasons: (1) they 
did not remain in the class during the entire school year; 
(2) they did not recieve a Spring Basic battery; (3) they did 
not have valid scores in both the fall and the spring on one of 
the tests; (4) they were missing data on sex, pre-school exper- 
ience, race or age; (5) they were in under-represented minority 
groups; (6) they were outlyers in terms of age. This seems 
like an extraordinarily low percentage of missing data-elimi- 
nated cases for a study of the size and complexity of the 1970- 
71 HSPV study. Another indication comes from the fact that 
there were 166 classes in the retained sites tested in the fall 
of 1970 and there are 166 classes retained in the analyses re- 
ported here. Thus, from the point of view of using the class- 
room as the unit of analysis, no data was lost — although the 
classroom aggregates were computed on less than the overall 



"58- 



possible nuiTiber of cases. One class, for example, had only 
3 eligible children, 3 classes had 4 eligible children, and 
3 classes 5 eligible children. Over, 70% of the classes, how- 
ever, had oyer 10 eligible children, and the average class 
size of eligible children was 13.46. 

IV Characteristics of the Analysis Sample 

A. Child Characteristics 

Table III-4 shows aggregate percentages and means for a 
variety of characteristics by site in the final analysis 
sample of Planned Variation children. Table III-5 shows the 
same data for the comparison sample of children. The child 
background characteristics shown are those which were found to 
have the strongest relationships to the test variables used in 
this report. They can be divided into three groups. The first 
group are child characteristics — specifically, age, race and 
sex. The second group are family background characteristics — 
family income, size of household, and extent of mother's edu- 
cation. The third group contains only one variable — the 
child's prior experience in preschools. Also shown in the 
tables are the number of children in the site and the testing 
level of the site. 

B. Classroom Characteristics 

Tables III-6A and III-7A show means of classroom means by 
site for selected variables in the PV and NPV analysis sample 
respectively. Two groups of variables are shbwn. The first 
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*Those means are not always based on n»s equal 
of children because of missing data. 
**This is hard to believe. 
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are site level means of classroom means / computed by equally 
weighting each of the classrooms in a site* For those vari** 
ables which are common tp Tables III--4 and 111-^5, the means of 
classroom means will vary slightly from the means computed on 
individuals, since the number of children per classroom varies 
within sites. By and large, however, inspection of the two 
sets of tables suggests that the differences are small. A set 
of variables not common to Tables III-4 and III-5 are included 
.in Tables III~6B and III-7B. These are variables which refer 
specifically to classrooms. In particular, we include here 
the percentage of white teachers, the percentage of certified 
^ teachers and the mean years of experience of the teachers in 
Head Start and of the teacher aidea. Also included are a mean 
index of the classroom levels of implementation in February 
and May, 1971, as seen by the sponsor and a meais rating of the 
srtaff working conditions by the teachers. Finally, the admi- 
nistrative arrangement of the Center is included (whether the 
Center is administratively run by a CAP agency or by the pub- 
lic schools) and where the center is located (in a public 
school or CAP location) . 

Although these variables are certainly not sufficient to 
paint a con^lete picture of the various sites and classrooms, 
they should give the reader some feeling of the x'ariations aisong 
sites on a number of classroom-relevant variables. 



mmmmmmmm 





V mft&mn-m& betw'cen the P lann ed Vari atioas and Goitsparisoa 

Tables IIX=S and IXI~7 indicate thafc tha Planned Varia- 
tA©ns aud Copparisen analysis saiJtples are# overall » essentially 
eqwivalent ©n Ki-^an Afc? in the classroom* ©esmal composition of 
the elBss£QCPs.r and or. the three weasiares of family background 
(Incesee, Household Si 2© and Mother's Education) . There are, 
h©fa'©veE, ©vera 11 da££©r©nces in the ethnic composition and the 
preschool eKp«£ience of children in "the two samples. Specific- 
ally, the sample has- proportionally fewer Slack and »re 
white children as veil as fewc-r • children' with pre-school ex- 
perience. The differences Letween t'm F¥ and comparison 
froup mmm are- not fally eliminated when only paired Planned 
Variation and coipparison sites are contrasted. Albany 10308} , 
for example, is the coiRparison site' paired in the design with 
Lafayette, The Pean classroep percentage of Black® in Albany 
is 08. n, while the aiean classroop percentage for LaFayette is 
only 22. g%, h rather large nwfcer ©f exas^le® liJse this could 
fee shown — even ffer the variables which ©how no overall dif- 
■ ferences is■^?tw«€^ll '^hr^ vimmed Vafiatiera anel C«®paris©r» analysis 

^mtmt way «iS *«r-fcifi« at •»ni?> difference m inputs between' 
^lhi«* F'y^ and tcr^p*3rs,;v&f; ^rc'^frti at., sofitrast the few© grcsaps --'i 
cles®rce-m '■^■i.zhifi r..^-StiH. t:sin*i the? -class.roop ap the unit of 
analysiE w; t-nif* pt^dc-U. «r«J ^ij^in-i fenly those sites which hav*i 
S^th r*V ^fs-rpas classes wsthif; thes*, wi- une^ a twr, w,*y 
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analysis of variance—- a model factor was crossed with PV/comparison. 
Table III-8 shows the amount of variation in the background and teacher 
characteristic -variables attributable to models, PV/comparison, 
interaction, and within cells* Four things should be noted 
about this table. First, for all but two of the variables 
(Percent Mexican-American and Household size) there are sta- 
tistically significant model to model differences. This sug- 
gests that the cofapoaition of the sites within models differ 
rather radically from model to model. Second, there are no 
overall statistically significant differences between the PV 
and comparison groups TaJcen as a whole for locations with 
both P¥ -and comparison classes # the P¥ and comparison groups 
am remarkably sijiiilar on the variables describad in these 
tables. Third, for only two variables are there statistically 
significant interactions between the models and the P¥/compar- 
isort factor in the table. This suggests that the P.V and compar- 
ison groups within modelB ar« remarkably ginailar. Fourth, it 
should be mentioned that most of the variation. on each of these 
variables lies within cells. Model to model variation plus 



rison variation plus the 



ariation attributed to inter 



actions betwetfr. the two ©ain lacto 



never accounts for more 



than 40% rsf the total variation and generally accounts for less 



than 3S% of the total varxatio: 




e-f FV and eorirjsr isdn gr©?ips w^rhin m^ieln will oo a Isnc? wa*/ 
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TABLE III-8 



Percentages of Variation: 

(1) Among models (both PV and comparison classrooms together) ; 

(2) Between PV and comparison groups pooled across models; 

(3) Due to interaction between models and PV/coroparison groups; 

(4) Within cells. 

Classrooms are the unit of analysis. The design is as un- 
weighted means crossed model by PV/coraparison using only 
those sites in the analysis sample which have both a PV and 
a comparison group of classes • The sum of the four sources 
of variation for each variable is 100%* 



PERCENTAGES OF VARIATION 



Variable 


(1) Araong 
Models 


(2) Bet. PV & ' 
Compar. €rps. 


(3) Inter- 
action 


(4) Within 
Cells 


Age 




38.5*** 

df=8 


0.18 
df=l 


0.29 
df=8 


61.03 

df=124 


Preschool 
Experience 




15.24** 

df«8 


0.50 
df=l 


15.29** 

df«8 


68.96 

dff=124 


Mexicans- 
American 




11.32 


0.06 

df=l 


: 0.83 

df=-8 


87,79 

df=sl24 


Black 
American 




29.57*** 

df===8 


0.66 
df=^l 


5.69 
df=-8 


64.07 

df==124 


Household 
Size 




8.82 i 


1.99 


12.97* 
df=8 


76.22 

df^.124 


Incomm 


: 


! 

18.9^*** ' 

d£'-b : 


0.04 


3.42 


77,64 

df^-^121 


Mother * s 

Education 




■ 

4 » X / ( 


0 * 05 i 


3,78 

df=^8 j 


92,00 

■■'j' . ^- i- X 


El/Ek 


1 


48. 58*** 

df^B : 

i 


0.01 ' 


0.06 ! 

i 




Pet. 


! 1S,24»* ! 

^ df 'ft i 

i I 
i * 


\ 

A f - - i 


15,29** i 

! 

: 

a 
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Table III-8 
(Cont'd) 



PERCENTAGES OF VARIATION 



Variable 



Teacher 

Headstart 

Experience 

Teacher 
Certification 

Staff Working 
Conditions 

Teacher Aide 
Year in HS 



(1) Among 
Models 



13.00* . 
df^rS 



19,35**^ 
df=ff 

16.69** 

df=8 

/ 

15.66** \ 
4f=8 J 



(2) Bet. PV & 
Compar. Grps. 



1.98 
df=l 



0.03 
df=l 

1.15 
df=l 

0.15 
df=l 



(3) Inter- 
action 



6.49 
df=8 



5.26 

df=8 

3.72 
df=8 

4.44 

df=8 



(4) Within 
Cells 



78.53 

df=108 



75.37 

df=lll 

78.44 

df=115 

79.75 

df=100 



* Statistically significant beyond the .05 level 
** Statistically significant beyond the .01 level 
**♦ Statistically sigruf'icant beyond the ,001 level 
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towards equalizing the PV and comparison groups on important 
input factors. A second implication is that the composition 
of classrooms within models differs dramatically from model 
to model and therefore that even pooling sites within models 
will not make the models equivalent on these variabj-eg* 

VI Pre-Test Score Differences Between the Planned Variations 
and Comparison Samples 

Still another way of looking at initial differences be- 
tween the PV and Comparison groups is to directly contrast the 
two groups on their pre-test scores. In order to give the 
reader a feel for the pre-test data we carried out these com- 
parisons in a number of ways. First, we show the overall mean 
differences in pre-test scores and their variances for the two 
groups. Second, we divide the children into twelve groups (by 
ethnicity, preschool experience and entering level) for each 
sample and present mean and variance differences for each of 
the twelve groups. Third, we move to the classroom level and 
show overall means and variances for the two groups on each of 
the tests. Fourth, we present the results of regression analyses 
using the pre-test scores as dependent variables, with a PV/cora- 
parison group dumxr.y variable* and a series of background charac- 
teristics as independent variables. Fifth and finally, we pre- 
sent results from a rr.altxvariate anolysis of variance with 
three pre-tcvst scores as dependent variables, and a aeries of 
background variablCfS &n covariat <;'S, for a design with PV/conipar- 
ison crofifjed wi th r-odc.-l<j. The overall conclusion fron^ thpf^^ 
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analyses is that after the introduction of a few controls there 
are almost no differences between the PV and comparison groups 
on the pre-test variables. 

Table III--9 below shows the simple contrast between the means 
and variances for the overall PV and comparison groups for five 
pre-test variables. Table III-9 indicates that there are a few 
significant differences between the overall PV and comparison 
groups on pre-test means and variances. Specifically, three 
of the ten statistical tests revealed differences at the *05 
level. There are no significant differences on the PSI, the 
Stanford-Binet and the Motor Inhibition tests. The variances 
for the PV and Comparison groups on the Book 3D and the Book 4A 
tests are statistically different, with the comparison group 
each time having the largest variance. It must be noted that while the 
variances for these two tests are significantly different, the 
ratios of the two sets of variances are very small — the large number 
of degrees of freedom made the statistical tests very sensitive to 
small differences* Finally there is a statistically significant 
difference between the overall PV and comparison means on the Book 
4A favoring the comparison group* Again, however, the differ-- 
ence is small ( roughly one tenth of the pooled standard devia-- 
tion) which indicatCB the power of a large sample in detecting 
small differences. We earlier pointed out clear overall dif- 
ferences between the samples in input characteristics which 
might be* causing these differences — specifically in the over-- 
«ii oercentage?> of children who arc* in their fir^t and second 




/ 
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Table III-9 

Differences between the PV and Comparison group : 
analysis sample in means and variances of 5 pre- 
test scores. Children are the unit of analysis. \ 
Only children with a valid pre- and post-test on \ 

the particular variable Jb(&iiigi.comf^red^wr^ u ^ 

in the analysis. 



Test variable 





Book 3D 


Book 4A 


PSI 


Stanford- 
Binet 


Motor 
Inhibition 


PV N 


1188 


1178 


1197 


389 


465 


Comparison N 


805 


803 


806 


297 


300 


PV mean 


11.851 


5.548 


35.498 


90.511 


5.047 


Comparison mean 


1^.103 


5.889 


35.835 


90.042 


5.116 


Difference be- 
tween PV and 
Comparison mean 


-0.252 


-0.341* 


-0.337 


0.469 


-0.069 


PV variance 


9.746 


10.063 


151.221 


184.289 


0.294 


Comparison 
variance 


11.092 


11.403 


142.014 


176.839 


0.274 


Ratio of PV and^ 
Comparison var- 
iance 


1.1381* 


1.1331* 


1.0648 


1.0421 


1.0729 



* statistically significant beyond the .05 level 



The largest variance was the numerator for this test. 
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year of pre-school. We might imagine that this difference 
(which favored the comparison children) could easily explain 
the few differences in pre-scores that we see on the uncon- 
trolled pooled samples. 

In a second set of analyses we disaggregated the children 
in the classrooms for both samples and divided each sample into 
twelve groups. Our stratification procedure took two levels 
of prior preschool experience (no and yes) , three ethnic cate- 
gories (Mexican-American, Caucasian and Black) and two levels 
of entering grade (El and EK) , We then separately compared the 
Planned Variation and Comparison samples for each of ten groups 
(two groups were left out due to sample sizes less than ten) on 
means for all four of the cognitive tests and the Motor Inhibition 
test (see Table III-IO) . This gave us a total of 50 indepen-- 
dent comparisons ; We found only 4 comparisons to be statistic-- 
ally significant beyond the .05 significance level — none were 
significant beyond the .01 level. Specifically we found that 

(1) white children with no preschool experient in Entering 
Kindergarten clashes in Planned Variations scored higher than 
their corresponding comparison group with Stanford-Binet ? 

(2) Black children with no preschool experience in Entering 
Kindergarten classes in the comparison sample scored higher than 
the corresponding Planned Variations group in both the Book 3D 
and the Stanf ord-lUnet tests; (?) Black children with preschool 
exporionco- in Entering first claases in the comparison vjroup 
scored higher than the carrosponding Planned Variations groyp on 
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Tible III-IO 



Prc-Tcst Mean Differences and Variance Ratios 
for 12 Groups of PV^d-CoBptrison Children 
- t>Tf T^Pre-Tosts*^ 



/ - 



/ 



/ 



>- 




Group 




Tests 






Ethnicity 

• 


Fro/ 

Sch. 

£xp« 


Ent. 

Gr. 


Mean 
Diff. 


4A 
Vsr, 
Ratio 


61 

Mean 
Diff. 


Var. 
Ratio 


PSl 

M«an 

Diff. 


V*r. 
Ratio 


Itoan 

Oiff. 


Vsr. 
Ratio 


HI 

Haan 

Oiff. 


Var. 
Ratic 




McXt*A»er. 


no 


El 


0.0S8 
df-109 


-1.21 


0.S22 
df-109 


.1.04 


1.291 
df'XOP 


4.20 


— 


... 


«0.056 
df»61 


4.50 




Mex*«Ajccr, 


no 


BK 


df*99 


U43 ' 


o.eo3 

df*104 


144 


-0.5S9 
df«104 


l.7fi 


df«3S 


1.17 


O.ISO 
df»20 


1.06 




Ifhite 


no 


El 




1*12 


«0-068 

df«lS6 


4.02 


4.207* 
df«155 


447 


•3. $11 
df«6S 


1.4« 


0.144 
df»101 


1.13 




White 


no 


EK 


df«416 


•1.06 


-0.209 
df«420 


4. OS 


1.772 
df«»424 


4.21 


J .S72 
df«14S 


-1.49' 


-0.006 
d£«167 


l.OS 




Slack 


no 


HI 


•0.699 
df«248 


1*06 


-0.290 
df-248 


1.07 


0*SSI 
df«2$0 


X.0< 


0.S7S 
df«127 


1.48 


-0.096 
df«114 


1 .63' 




Black 


no 


E)C 


-O.OSS 


-K04 


-0.657* 
df#SS$ 


1.19 


'0,629 
df»S$9 




•4.195" 
df»I62 


*1.04 


^•0.178 
df«154 


4.27 






yet 


EI 












— 


.... 










H«x»^Aiitr« 


yea 


EK 
























«hlt« 




El 


df»6t 


-U0$ 


«0,46a 
df«6? 


1.07 


df»67 


4. IS 


-1.512 
df*2f 


*1*S$ 


*oaos 

dfMS 


l.SO 






yet 


Etc 


df*?2 


-K26 


-0.?92 
df*7JS 


1.04 V 


-3.a72 

df-7:$ 


1*94* 


5.179 
df»25 


1.01 


0.001 
df«51 


-1.7$ 




suck 


yes 


El 


df*ja9 


-1.24 


0,532 
df*l^> 


K29 


-^1*43? 
dr*l09 


AM 


0,625 
df»46 




dC*48 






0Uck 




HK 


4f*lU 


-1.19 

i 


•e.26S 
df*U2 


l.69» 

i -J 


-2.312 

df»ns 


1.02 


$.922 
df*27 


*4.ii* 


-0,29S 
df^l4 





^ All dlffer^ncef. arts expressed vuh the PV frcup* as poiitlve and the C«m|?ayison 
group as negative Si'^^ilarly, the variance ratios, whlla all greater thani l.M^ 
are given a sign * p^siitlve sign Indicates, that tha P¥ variance is larger sj5d 
a nejtttlve that the Co«|?ari*oo gr^r^p is larfar* 

♦ statuticaUy slgnteicant as t^t .OS U\t\ 
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Book 4A. No differences in means were found for the Motor 
Inhibition test. 

The large nusiber of statistical tests carried out and 
the very few number of statistically significant results leads ^ 
us to suspect that these differences might have bcqurred by 
chance. The fact, however, that two of the statistically sig-, 
nificant findings occurred for one group, and that the tendency 
for the other tests for these groups to go in the same direc- 
tion for all three tests though the t*s are less than one, sugnests 
that there may be a difference between the comparisons and 
Planned Variation samples for Black children with no preschool 
experience in Entering Kindergarten classes. Overall, though, 
a general cohclusion is that the Planned Variation and Comparison 
groups are essentially similar on pre-test mean scores. 

We then compared the variances for the groups. Five of 
the fifty variance ratios were statistically significant at 
that «05 level, with two of the five having greater variances 
in the comparison grotjp. There doesn't B&m to be any pattern 
to the differences in variance. Book 4A is the only test for 
which there is not significant difference in variances. For 
Black© with prior preschool and EK,^ where there are two statistic- 
ally signSi'icant variarxe ratios, the Coiaparisen greup has the 
larger variatioi* for one aroup and the smaller for the otiier. 
Since no patterns «txist and tht^re are only live sigmfieant dif- 
ferences, none reachir.cj a highc^r IqvqI ©£ sign-if icancc than ^©S, 
our conclusion about difforencea arena variances is the* sai?A» as 
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our conclusion about differences among means — that the PV and 
Comparison groups are overall very similar. 

The third approach was to contrast, classroom means and 
variances for the overall VV and Comparison samples » These data 
are fhovsrn in Table III ~11. There are no statistically signi*- 
ficant differences between the means and variances of the over-- 
all PV and Comparison groups on any of the five pre-test vari-- 
ables* 

Although no differences exist between classroom means in 
an uncontrolled state ^ t^ere may be differences after controlling 
for some relevant variables* This situation was tested in the 
final two analyses presented here* In the first analysis^ presen 
ted inTabJe 1X1-12, wa ran a sat of five regressions with the 
five pr^'-test variables m the dependent variables* In each 
^quation -the variable of principal interest was a PV/Comparison 
dUM^y vairiaMt* Thera;-w#re 7 major control variables — Mean 
Claj^sroop- Age I- Percent Mexican^-I^er leant Percent Blacky Percent 
with Previous Preschdol gxparienca^ Mean Income ^ Mean HoisEehold 
Sisie and Mean Mother Edi^^option* In in^ equaiioiij did. the duumy 
vailS^iifele' representing M€sirt^ersh:^.Sr in P¥ ox ^omparisoi^ group 
enter with. a statistically* sign! ficas^fe"- coefficient** 

. The fifth and final .at t^ippt -fc© set i^hether there* are s£g- 

We, also ran thesa ^q^ati^ns l€?ttiftf there bc-'dif far#nt slopes 
for thi^ control variables' Mr thc^PV* m4 "eowparison' grmsps, tfmt. 
!^#re no serxous diffff^renc^s feeti^^rmn tte presented 'r^imlts and the* 
rastiits of -thase runs*' ' In parficiilar^ in no iitstanc^ was tht 
dymy ■ P¥/copparxson vari0li4^. fetatistxoallY significant* 
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. Table III-ll 

Differences between the PV and Comparison i&tai 
analysis sample in overall means and variance on 
each of five,^ pre- tests. Classrooms are the unit 
of analysis. Only classrooms with valid pre- 
and post-test scores on the particular variable 
being coifpared are used in Uie analysis.*^ 









Test 


Variable 






Book 3D 


Boole 4a 


PS I 


Stahford- 
Sinet 


1 Motor 
Inhibition 


PV N 


101 


101 


101 


$1 


/ 


Comparison N 


^ 65 


65 


65 


• 47 . 


: 


PV Mean 


11*792 


S.S72 


35.081 


90.591 


, S.O02 


Cojnparison mean ' 


11,993 


S,?88 


3S.384 : 




5.660 


Diff* between 
PV and compa^ 
risen mean 


-0.201 




: -0.303 


0.292- 


-0.058 


I*¥ variance 


2.981 


2,263 


54.194 


66,3:5f 




Contparison 
variance 


2.436 


2.5 34 


47..4B4 


54,466 


0.184 


Ratio of PV and 
epmparison 
variance 




1.1197, 


•1.1413 


1.2183 


X # -2340' 



^ There ars no statistically significant dif>ferences in in&an os 
variance '©a tween the PV and "Comparison groups. 
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%tMndmdxiQd and rat* r^js-essum ieeffissenti- tm- & fVfCQVfMi'mn gmvp 
»e!fflb<frshs|» dx^^f varsablfr fer fsve ^rt'- tests* Ife© ©veyall elassff@sa 
Deans analysis sar.plte^ waf, uss'd.. Scveri e©r«tf®l varsablcts were als© in 
the ©quatioR C?*©ycW,t fre^.-4©ti!?. rn-^schs®!; „ Percent ^?«,».KiC:if5=.\«"^sftie.a;t» 
Percent Black, n&m Ag«f« Mean IncoR©, ^?ean M®us©h©W Sase and .Heai? 
J-tother " s Educa t i ©stl ■ ^ 



Test 




ai?4 lesi var- 
iable 



iegressa@B Co- 
efficients 



If 



S3i 



art 



Itaradard 
raw 



of Varid53C© 



» Ml 



Ml 




*•* Statist st.il ly ':Sg?ijf ;4-3!5it l^^ycs^iS ts.« -COi levf^. 



^ !^.<^;>^-.r^ ^rt::^^^ ^te^^^^v fee t-#!iai-;^?> %tW:y ^^^m ^^mi • m 

^.rt:^thm^ with rV/'x r oj>' Sacs^-r w^Ml^^ a c^^ccsfud 
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one*-half of the children have reached criterion on each 

of the subtests at post -test time. For the entering first grade 

children who have attended preschool, the percentage reaching 

criterion is closer to 75% on the average though it must 

be noted that the percentages for these children are initially 

quite e* bit higher than for the other children, o)tQe^again^ 

there is a clear pattern of greater gains for the El children 

with a l^ss consistent pattetn of greater changes for the 

children wno have not had a iprior preschool experience. 

\ 

1*he use of percentages bf children reaching criterion 
is an exploratory and descriptive way of presenting the data 
gathered in this study. By and large, its utility rests 
on the adequacy of the criterion and on the feel for the data 
that it gives to the reader. There are a number pf statisti- 
cal tests that we might have applied to these data but since 

_our principal purpose was descriptive and our criteria arbi-- 

trary, we felt that applying inferential statistics might not 
be justified. / 

II. Estimated Effects of the Head Start Experience 

This $ectron atcempts to estimate changes attributable 
to the Head Start experience. In Table "gains" are 

expressed in terms of a difference between an actual and an 
expected score on the particular test. The "^expected" score 
is intended to represent the score the child would have 
received had he not attended Head Start. Since an appropriate 
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OycriiXX Gain for children in the Planned /Var^Ciiion 
Study* in 1570-71. **C:mv.s'' nro cc-.r ;-,tod ybv Kubtr.nctlr^ 

to estimate mean **^ains,'*) / 

/ 



Gro 


u\ 


jc; 

is* *tt 


c ^ 


PS I 


Booh 3D 
"Cain" 


Book 4 A 


X4ihib, 


Ston^orcl- 
Dinct 


1 




No 


i 

£1 


Gain«a 
6.7 

tT^ 111 


1*8 


2.6 

111 


/o^l4» 

63 




, 2 




No 


Ek 


0,40'* 


0.2* 


1.9 


0.19 


9.3 














4,7 


22 


40 


3 




No 


EI 


4.7 


1*1 


ft 1 4 


ft A 










157 


158^ 


158 


103 


65 




Whte. 


No 


Ek 


4.0 


1.1 


1.9 


0,28 


7.0 










426 


422 


418 


169; 


147 


5 


Blk. 


No 


EI 


252 


2, A 


4*8/ 
" ■ '■ / *ittf\ 


0.40 

1j6 


111, 
'l29 


6 


Blk, 


No 


Ek 


9*4 

561 


1.0 

557 


1 

lv9 

7 «^ o 


0.38 

136 


10,4 

164 


7 


M-A 


yes 


El 










— — 


S 8 




yes 


Sk 













9 


Whte. 


yes 


El 


7.1 


* 

0.4* 


3.4 


0,05* 


12.2 












69 ' 


69 


47 


31 


'0 


Whte. 


yes 


Ek 




2.4 


2.7 


0.47 


14.5 










75 


75 


74 


33 


27 


' 1 


Blk. 


yes 


El 


7,3 


1.7 

111 


4.8 


0.03* 


8.4 










111 


111 


50 


48 


^2 


Blk, 


yes 


Ek 


-0.7* 

115 


-0.2* 


1.1 


0.47 


-2, a* 










114 


113 


16 


29 








TOTAL 




1.2 


2.7 


0.26 


8.7 








S£>gain'^5,2 


1993 

2.8 


1981 

4*5 


765 

0.56 


686 

12.2 











* Jndicntoc gain not stntlfitically clgniflcanUy greater 
, thnn ?.cro beyond the ,05 level. f ^ . 
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control group did not exist in the study — that is^ a group of 
children v;ho did not attend Head Start — we had to estimate 
expected ir.aturational growth by using age variations in 
the children in the study. Briefly, the pxoceduire was 
as follows. First, the children were divided into the 12 
groups represented in the tables. Second, the pre-^test 
scores of these children on all of the five test variables 
were used as dependent variables in 60 separate regression 
equations (1 for each of the 5 pre-test variables for 
each of the 12 groups). This gave us the relationships 
among the test score and the independent variables prior 
to the time the children entered Head Start, The ;Lndepend-- 
ent variables for each analysis were age, sex, family in- 
come, household size, mothers' education and appropriate 
dummy variables to control for missing data. By using the ^ 
coefficients for these equations and the original data, ^ 
we arrived at an "expected" pre-score for each child. 
Within each of the 12 sub-groups, the mean of the expected 
pre-scores equa.ls the mean of the actual pre-scores. 

The regression analysis estimates the effect of age 

with controls for the three stratifying variables and 

i ♦ 

their interactions as well as for the other variables in- 
the regression jeq.uatiofts (sex and family background) • 
Therefore, witH relspect to the relationship between age • 
and the score on a particular test, it can be argued that 
the coefficient for age (expressed as the average change 



per month in a group) in each equation reflects the rate 
of growth for the children in each group prior to their 
entering Head Start. (The assumptions for this arguinent 
are discussed in later paragraphs). In other words, the 
average expected difference between a child at 48 months 
and at 56 months without Head Start is reflected in the 
coefficient for age for his particular group. We then 
can estimate what we would expect a child's score on a 
test to be 7 months after entering a Head Start program 
if the program has no effect at all. This assumes that 
the relationship between the test scares and the independ- 
ent variables remains during the Head Start program as it 
was prior to the Head Start experience. 

The analys'is required two more steps: estimating an 
expected post-test score 'and finding the difference between 
the expeoted and observed post-test scores. First, for 
each child in each group on all five tests we estimated an 
expected post-test score by adding to his pre-test score 
the product of the number of months he was in the program 
and the age coefficient for his group. This expected 
post-test score reflects an estimate of the child's change 
assuming no Head Start effect. We then su'atracted this 
.expected post-test score from his observed post-test score 
and computed group means. The mean differences between 
expected and observed post-test scores are then interpreted 
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as the effect of Head Start above and beyond the effect 
expected by maturation alone. 

In somewhat more precise terms the procedure was as 
follows: (1) Divide the children into "the 12 "groups. 
Consider now one group (Black children with no prior 
preschool experience who will enter public school kinder- 
garten) and one test (the PSI) . The procedure for this 
group and tost was the same as for all other tests and 
groups. '(2) A simple linear regression using PSI pre- 
score as the dependent variable on age^.a^, size of house- 
hold, income of family, mothers' education, a dummy variable 
for missing cases on mothers" education, a dummy variable 
for missing cases on income, and a dummy variable for 
missing cases on household size. Complete data were avail- . 
able for age, sex and PSI pre-score. Following Cohen (1970), 
the other independent variables were given their observed 
values unless there was no information for the child on a 
variable. In this • instance , the variable was given the value 
of zero. Dummy variables were then computed for each of 
the three variables with missing values. The^ dummy variables 
were coded with a 1 if the data were missing and a zero if 
the data were present in the original variable. Thus, if a 
child had a value for mothers" education, he would be 
assigned that value on his variable "mothers • education" and 
a zero on the 'mothers' education" dummy variable. if, on the 
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other hand, he had no observed value for mothers' education 
he would be assigned a zero on the mothers' education 
viLriable and a one on the mothers' education dummy variable. 
For the group and test we are considering, the final 
equation was, as follows : 

PSI pre = bo + Bage*Age + bsex*Sex + bHHsi2e*HHsize + 

bi*Income = bi^*Mothers' Education -t- bMED*Mothers • 
, Education Dummy + bjD*Income duimny + bHHi5*Household 
Size Dummy 

the "expected PSI pre-score" for a child equals a 
constant (bQ) plus a coefficient for age (b, \ times the 

cig6 • 

child's age plus a coefficient for sex (b ) times the 

S 6 X 

child's sex (l=male/2= female) .plus a coefficient for size 
of household (bHjjgj^^g) times the number of persons in the 
child's household plus a coefficient for the child's 
family income (bj) times the family income etc. The 
coefficients for the group of 620 children were: 

«,* - • 

bo = -32.606 b^ge = 0.7652*** h^^^ = 1.9911* 

^HHsize = 0.0119 become = 0.0738*** b^E = 1.4338*** 

bMED = 18.2855*** hj^ = -1.2671 bHHD = "0.8753 

One * indicates statistical significance at the .05 level; 

Three *'s indicates statistical significance' at the .001 level. 

ErJc ' '^2 
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The equations were run on all children in the pre- 
test analysis sample v/ithin a group. The key to the genera- 
tion of an expected post-test score for a child is in the' 
coefficient for age (here it is 0.7652 and is statisti- 

* 

cally significant beyond the .001 level). The interpreta- 
tion of this coefficient is that on the average for this 
group, children's scores increase by 0.7652 points for 
every tnonth of age. In other words, in this sample, 
children who are 60 months old; score 5 x 0.7652 or 3.826 
points higher than^ children who are 55 months old. If we . 
assume that other things are equal then a child's score 
would increase naturally over the period of time that he is 
attending preschool — specifically, it would increase 

if 

naturally 0.7652 points per month while he is attending 
Head Start. 

Granting the assumptions that other things are roughly 
equal and that the relationships hold for the various age 
levels, we can compute an expected post--test score for .each 
child — a test score which reflects only natural growth 
and does not reflect the Head Start intervention. To do 
this we calculated, for each child, the number of months 
between pre and post-test. We then took his pre-test score 
and added to it the nuirtDer of months between pre and post-test 
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times the coefficient for age {0.7652). For analyses 
'which retain the original twelve group composition, we 
could have used the alternative procedure of taking a 
child's "expected" pre- test score and adding ,to it the 
product of the number of months he was in the program 
times the coefficient for age. Since 'there was roughly 
8+ months, between pre and post-test time, the average gain 
attributable to natural maturation was roughly 6.4 points. 
This procedure was carried out for each of the children in 
the group with a valid post-test score (561 children). 
Each of these children's predicted p^^st-test scores was 
then subtracted from his observed post-test score and a 
mean for the entire group of children who had both valid' ' 
pre and post-test scores was calculated. Group means for 
, these differences were then calculated and the results 
are presented in Table TV-4. 

A number of things should be noted about Table IV-4. 
First, almost all of the estimated gains in Ta ble IV-4 
are statistical ly significantly greater than zero. Only 
nine of the forty-nine comparisons shown in the Table do 
not reach significance. Second, for the PSI and Book 3D tests , 
the estimated "gains" attributable to Head Start (see Table l v-4) 
a re roughly one-half the total gains shown in Table IV-1 . This' 

.^indicates that one-half of the totar.^ain__ is estimated to be 

attributable to Head Start while the other half is attributable 
^ to maturation. Thus, in effect, the children double their rate 
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of growth on these tests during their months in Head Start. 
For the Book 4A and Motor Inhibition tests the Head Start 
experience accounts for roughly 70% of the total gain. For 
these tests' the children are tripling their rate of growth 
during Head Start. 



Third, by and largo, the estimated gains shown in 
Table IV-4 f o r the Stanf ord-Binet are greater than the gains 
in Table IV-1. On the average,' the estimated gains are 85% 
larger than the actual gains. This indicates that the 
coefficients for age for the Stan ford-Bine t are generally 
negative. In other words, older children at pre-test time 
on the average have lower Stanfdrd-Binet scores than younger 
children. If the assumptions for this estimation procedure 
hold, then, it appears as if Head Start arrests a deteriora- 
tion in Stanf ord-Binet scores and additionally accelerates 
the rate of growth of Mental Age as assessed by the Binet. 
The arresting plus the acceler^.tion appears to be on the 
order of two-thirds of a standard deviation. 

Fourth, there seem to be no consistent differepices , 
in estimated gains between children with and without ' 
a prior preschool experience. Fifth, there are greater 
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estimated cams for £1 chx ldren_ on the PSI^ Book 3D and 
Book 4 A test s > There are no dxfferonces between £1 and Ek 
children on the Stanfor d Bmot and £k children tend to gam 
more on the Motor Inhibition* Sixths there seem to be no 
consistent patterns of differential gains for the three 
ethnic groups. 

A variety of assumptions wcro made in this analysis. 
First, we have no way of controllmg for the effect of pre-- 
test sensitization on the children. It may be that the 
specific effect of taking the pre-^test contributes to the 
post- test scores Second^ we hiave to make the - assumption 
that there is no differential selection of older and younger 
children within groups tha| is^ we nust assunte, for 
exait^^le, that the older children in a group were not more 
nor less clever than the younger children* There is no v;ay 
of controlling for this. Third, we must make the assumption 
that the coefficients for age ,are unbiased* We have no 
assurance of this aside from the fact that we have physically 
controlled for ethnicity / preschool experiences, and entering 
grade and their interactions as well as the variables in the 
equation. Yet even these rather' extensive controls do not 
assure that the age coefficients are unbiased. 

If there were px'e-test sensitization, the ^'estimated 
gains" in Ta>>le IV-4 would be overestimated. If there v/ere 
selection effects into Head Start programs favoring more 
Clever younger children ahd less clever older children, the 
'•estimated gains" would again be overestimated. If, however. 



1 

the selection procedure operated in the oth«r direction, 
the estiniatsid gams would ,be underestimated. Bias in the 
age coef f icjLent.s could lead to cither under or over estima- 
tion of "gams". Our best guess is that the coitJbin.ation of 
these influences probably leads to a slight .overestimate 
of the "gains" shown xn Table IV~4. Yet even if the 

gams were halved the overall increased" growth rate- would 
still be^on the order of 25% for the PSI and Book 3D tests, over 
331 for the Book 4A and the >^otor Inhibitaoa, tests and the 
natural loss on the Stanf ord-Binet would be arrested* 

Interpretations and Conclusions; 

The two central purpose^ of this chapter were firsts to 
describe the overall changes in test scores for the total 
sample of Head Start children and second, to estimate to 
what degree the ^changes can be attributed to the Head Start 
experience. Data, suminarixing these efforts are contained 
in Chapter IV- 5* Coluum 1 of that table shows the average 
total gain for children in the overall analysis sample for 
the five outcome measures* Column 2 shows the portion of the 
total gain attributable to natural maturation (the estimated 
amount of gain that would have occurred had the children 
not been in Head Start) • Column 3 shows the estimated 
amount of gain attributable to Head Start* All estimates in 
in this table are expressed in standard deviations of the 
pre-scores of the tests. 
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TABLE IV- S 



Gains fpr thfj t:btai anal.ysx.s saif^ple on S oatconta 
measures, ObBBtved gains are partitioned into two 
components gams attributable^ to iT?atoration and 
gaiBS', attributable to an Head Start experience , , 
All gains aro expresg'ed m i.ndi-vidiial 1#V0.1 pra«- 
test standard •deviations. 
\ 

! 



Test 


Observed 

gain 
(total) 


Attrxbas^lff . to . 
roataration 
Cestimatedl 


Attributable to 
Head Start 
{estinsatedl 


PS I 


0*942 


0.496 




Book 3D 


. 0.727 


0.363 


0,364 


Book 4i^. 


1*151 


0, 333; 


0,818 „ 1 


Motor 

Inhibition 


0, 36 


0.10 


0.26 1 


Stanford- 
Binet 


0.34B- 1 

j 




0.644 1 



Conclusions f^om.this table are 3,SraigLcforward. 
Obst^rvttd gains for the five tests varied from a low of 
about 0.35 standard df£ViationB\pn the S tan for^i -Bine t to 
I4I5 standard devxations on the Eo'ok 4A test* In all 
instances the <gai;v- attributable to the Head Start exper- 
ienct* indicated that diiring Head Start- the children at 
• least double ' their aoriwai rate of growth, F^r the PSI and 
Book 3b tests the total g^ins are estimated to be evenly 
divided faetveen m,%t.\xtation and the Head Start experience. 
For the Book 4a and the Motor Inhibition tests the Head 
Start ©xperitjac© accounts for over two third^? of t'he total 
gains. Finally^ the St#nford-Binat. the estimates 

xndieato' that th€ Iteaa Start experience arrested a decline 
of roughly 0.3 titar^oiard ctevxatxons and additionally increased 
ehiMr^in^s BCotr^s\hy another st^n&atd deviations. 

The text, of the chapte-r^^nnts^ oat a varietv of untested 
- assumptions ■underlying thh procedures .Jjsci to reach these 

0Stjj«ates.; suspect that thr- procedurcfs pay havu produced 
■ slight ovor(-,tint^tcf% in the vtii-ct ot the {fead Start exper- 
ience* Yatt even halvir.n: tJvy gains attributable to Head Start 
vfould reauli xn iff.'-c-;, of * substantial mgrdtude iiidicating 
at least t;s<:- powt-rtj.: :.hc'ti tvrrs t-.n'cct t^f iiead Start on 
th© sie-asurc'd out c o:n »;;■»; . ■ ■ , 

In the' course o£ r#-'achim; taesi? estamates threo othdr 
issues wetir addresj-cd -in this chapter. Thtv fxr§t wet of 
issuea had to do witi'i; di t'fc-r*' : ra* yxans by a Cew dj.f ferent 



ER^i.ypes." ©-f children in the aaisple. Specifically we found thati 



Children with prio^ Head Start experience averaged lower 
overall gains than children without prior Head Start 
experience. Thus, the overall effect of a second 
year of Head Start seems to be less tha^ the effect 
of the first year* An indication that we must be 
.careful making this inference stems from the fact 
that oyx estimates of the gains attributable to 
Head Start for the children enrolled in a first 
and second year of preschpol^eem to be roughly 
equal (see Table IV--4) * Thus, the difference in 
overall gains for the two groups may be attributed 
to differences in the gains expected from natural 
maturation. Our interpretation of this is that the 
first year of Head Start acted as an homogenizing 
experience on children {at least with regard to 
measured outcomes) . In our analysis such an effect 
would reduce the differences between the prescores 
of children of different ages who have had a prior 
Head Start experience thereby lowering our estimates 
of the rate of growth that such children would have 
had without a second year of Head Start, 

Chxldron v;ho will enter first grade directly from 
Head Start tend to gain more overall than children who 
will enter kindergarten though the effect is pronounced 
only for the Book 4A. When the gains attributable to 



the Head Start experience are considered it appears 

as if children in El siter? profit more from Head 

Start on the PSI, Book 3D and Book 4A. We can 

speculate that, two things are occurring here. First, 

teachers of children iTi'^El sites may feel a strong 

obligation to prepare their children academically 

while teachers of future kindergarten children may 

not feel such ^n obligation. Second, since the 

children who will enter first grade are generally 

older than the children who v/ill enter kindergarten 
- * ■ - ~ 

the effect may simply be due to^gre^ter maturational 

readiness for instruction. 

There are no differences between El and Ek 

♦ ■ ... 

children on the Stanford-Binet while the Ek chidlren 
tend to gain more on the Motor Inhibition test. 

3. There are no discernable patterns of differences 

among the gains for the t^hree ethnic groups studied* 
here. 

The second set of issues briefly addressed in this chapter 
has to do with ways of presenting gains. Tables IV-2 and IV~3 
present data for the Book 4A and. Book 3D subtests structured 
as criterion referenced tasks. The procedure is exploratory. 

Finally, we examined the pre and post-test varianc(is for 
each of the five outcome measures used in the study. With one 
exception the post-test variance was smaller than the pre--test 
variance indi^cating that the fan spread hypothesis is probably 
incorrect for these data. 




Chapter V 
SOME METHODOLOGICAL CONSIDERATIONS 



c 



Introduction ; 

For many "true" experiments there is a clearcut 
"best" method for analysis. The analysis strategy flows 
logically from the structure of the experimental design 
and the hypotheses of the experimenter. When, however, 
the experimental design is compromised as is Planned 
Variation's, the choice of an analysis strategy becomes 
less obvious. This chapter considers issues in selecting 
analysis strategies for the Planned Variation study. 
It is divided into three parts. The first part discusses 
what unit of analysis is appropriate. The second part 
considers strategies for reducing bias in estimates of 
differences between groups. The third part describes 
three analysis procedures used in later chapters of this 
report. 

I. The Choice of a Unit of Analysis 

One issue prior to the selection of appropriate 
analysis strategies" is that of choosing a unit of analyst 
In this study the choice is among models, sites, class- ^ 
rooms, and children. Three considerations in making tjis 
decision are: practical considerations (what is needed 
to answer certain questions) ; constraints imposed by the 
experimental design; and the conceptual framework (how 



the application of the treatment is perceived) . These 
considerations are discussed below. 

(1) Fir^t, we wanted to select a unit of aralysis 
that would common to most of the questions we were 
asking. The ipossible units of analysis are models , sit^s , 
classrooms arid children. For each of these we could disi- 

i ^ . ^ ■ 1 

tinguish amonjg PV and comparison groups and thus any | 
could be used for the analyses presented in Chapter VI ^ 
(analyses contrasting overall PV and comparison effects). 
*^But for analyses presented in Chapter VII (comparisons 
among curricula) , we could not use the model as the unit 
of analysis since we would have no error term for testing 
the significance of differences among the models. Thus, 
we needed to choose among sites, classrooms and children. 

(2) A second consideration had to do with the 
sampling design used in the study. In order to obtain 
estimates of the' variability of model to model differences, 
we need more than one observation for each model. The 
natural level of replication in this design is the site; 
however, there are two serious problems with this 
choice. First, the original design was conceived on a 
three level nested design (sit;es within models, classrooms- 
within sites, and children within classrooms; the PV/ 
comparison factor woul<3 cross sites within models) . 
Theoretically, in this' design, 3ites woulfl be a random 
factor, and variatioi) among sites within models would be 
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the appropriate error term for testing the variation among 
models. However, since sites were not randomly assigned 
within models, they cannot technically be considered a 
random factor. And as our analyses in Chapter III 
indicated, sites cannot even be argued to approximate a j 

random factor since sites within some models are clearly | 

I 

differ&^t from sites within other models. Unfortunately, J 

ji 

' i 
this argument applies to classrooms and children as well, I 

since heither involved random assignment,* 1 

5! 

\ ?; 
The\second problem relates specifically to sites. 

Two models" have no replication at the site level and six ; 
models haVe no site replication for the Level III testing, '{ 
The lack of replication of sites for some models leaves 
us without an estimate of the error term for those models. 
Although it might be argued that we should limit our 
analyses only to those models which have replications, we 
decided rather to note the problem and temper our conclu- 
sions about the effects of the models rather than to eliminate 
them from the analyses. Since there were no compelling 



*The point is that there is no intrinsic reason in the original 
sampling procedure for choosing sites as the appropriate unit 
of analysis over classrooms or individuals. In order to make 
inferential assessments of model to model differences, we must 
make the assumption that the chosed unit analysis was a random 
factor--that the sample of sites , classrooms or children was 
randomly drawn from som^ larger sample. Given the sampling 
procedure there is no reason to select any of the possible units 
as more closely approximating a random factor than any other. 
For those readers wishing to contend that inferential statistics 
cannot be employed without a clear indication that our unit of 
\ analysis is a random factor, we suggest that they use the 
significance testing as an heuristic device. 
\ 
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reasons associated with the design, We therefore 
ruled- out the selection of sites as the unit. 

This leaves us with either the classroom of the 
individual as the appropriate unit. Two 'arguments con- 
vinced us that classrooms were desirable. First , one of 
the problems faced in any experiment of this sort is the 
problem of fallible data. Of particular conceri) here \ 
is the "reliability of the various independent measures 
(the reliability of the dependent variables is of less 
concern) , If we use the individual as the unit of analysis 
there is a considerable amount of error in the assess- 
ment of any of the background and pre-test characteristics 
of the child. For some measures, there is little error 
(sex, race and age are examples). For other measures, 
the reliability ranges from roughly 0.65 to 0.90 depend- 
ing on the characteristic. If we move to the classroom 
as the unit of analysis we aggregate individual observa- 
tions. Given the assumption that the errors of measurement 
are randomly distributed with a mean of zero, the aggrega- 
tion should serve to cancel out some of \ the error and 
make our measures more reliable. By and\, large at the 
classroom level the reliability of our .measures can be 
estimated to range from roughly 0.85 to 0.99— a substantial 
improvement over a range from 0.65 to 0.90* 

♦Take for example the PSI pre-test and the Book 3b pre-test 
In Chapter II v;e indicated that their respective reliabilit 
for individuals were roughly 0.90 and 0.70. * Roughly 37% of 
the PSI pre-test variance lie? between classes and roughly 
26% of the Book 3D variance lies between classes. If we 
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The high reliability estimates obtained by aggregating 

into classroom means gives us the advantage in our analysis 

I 

of not having to correct our independent variables by 
the reliability coefficients. 

The second concern that led us to classrooms as the 
appropriate unit stemmed from our desire to use a number 
of measures collected on the classroom as the unit. 
These included teacher and teacher aide characteristics , 
and estimates of the degree to which the classes were 
implemented. Had we used the child as the unit of analysis 
we would have been seriously overestimating the number 
of degrees. of freedom available for these variables. 

(3) On conceptual as well as statistical grounds 
it seems reasonable to select the classroom as the appro- 
priate unit. By and large, a child's experience in Head 
Start is confined to one classroom, one teacher and one 
teacher aide. There is a great deal of variation within 
sites in the characteristic of the teachers and aides, and 
so it might be argued that children in different class- 
assume the class size to be constant and roughly 12 (a bit 
of an underestimate) we can use Shaycroft^'s (1962) . formula 
to estimate the respective classroom reliabilities as roughly 
0.98 for the PSI and roughly 0.90 for the Book 3D. , Shay- 
croft's formula is: -^""^aa s 2 where 

r^ = 1 - -) (—^2—^ 

r— is the estimated group selectivity, r is the estimated 
inaividual reliability, n = the number of Shildren in the 
classes, s^^ = the 'variance for individuals, and s— ^ = the 
variance for the classrooms. ^ 
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rooras within sites undergo different experiences. This 
argument is strengthened when we recall the tremendous 
variation within sites in the sponsor's estimates of the 
level of implementation. Although it might also be rightly" 
argued that different children within the same classroom 
undergo different experiences and therefore should be 

« 

treated separately in an analysis, We have no information 
on what causes these differences (as we might have if we 
had carried out classroom observations on individual 
children) . Thus there seems little purpose in not aggregat- 
ing the children to the classroom level whfere we might be 
able to distinguish among group experiences. 

Overall, then, the decision was to choose the class- 
room as the unit of analysis. One byproduct of this deci- 
sion over the choice of the site as the unit was to increase 
the number of degrees of .freedom that we had to work with, 
thereby allowing more control variables to be entered in • 
our analysis. In this sense the choice of the classroom 
represents a compromise between the site and the individual— 
with the individual ostensibly giving us the most degrees 
of freedom to play with and the site giving us the fewest. 
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Reduction of Bias 

In Chapters VI and VII we are concerned with the prob- 
lem of comparing groups on a nxunber of outcome measures • 
For reasons discussed above most analyses use the classroom 
as the unit of analysis. Ideally, we would have wanted 
classrooms randomly assigned to groups.^ Jlandomization 
would have insured that the probability that groups dif- 
fered initially on any variable, measured or unmeasured , 
was small. Thus we would have had confidence in the 
results of direct comparisons on outcome measures because 
we could assume that the groups differed on treatment only — 
comparisons among groups could be assumed to be unbiased. 

Unfortunately, sites were not randomly assigned to 

models and classrooms within sites were not randomly 

m 

assigned to PV and comparison status. Data presented in 
Chapters II and III demonstrate that the composition of 
sites within models and of PV and comparison classrooms 
within sites differ in a number of possibly important ways. 
In the absence o^ randomization, no statistical method can 
control for all possible variables which may influence the 
outcome meas-ure. If we can isolate and measure those 
variables* which seem important, however, we can attempt 
to control for, biases using a variety of analysis 
strategies. 

Our approach to choosing an analysis strategy was 
agnostic. We don't know the "best way" to answer the 
questions addressed in this report. Thus we present data 
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♦Such variables are called concomitant variables or covariates 
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from a number of analyses which use different methods of 
controlling for possible biases in the data. Different 
analyses, however, often lead to somewhat different 
estimates of the effects in which we are interested. Some 
estimated effects are consistent across different analyses 
and are therefore quite compelling. Others are more 
sensitive to the nature of^he analysis and are therefore 
less compelling^ though of ten] suggestive. One result of 
this approach is to give us rADugh "confidence intervals^* 
for the sensitivity of estimates to different analytic 
approaches. 

The data used to compare groups in this study consist 
of pre- and post-test scores, background characteristics;,'^ 
and teacher and site characteristics. In comparing groups 
on post-test scores we generally want to control for as. 
many important differences among the groups as possible.* 
Three approaches are taken here to control for differences: 
cross- tabulation , covariance and matching. 



*The availability of pre-test scores as concomitant vari- 
ables is a great advantage r but it is not at all clear 
how to handle them. We mav simply treat a pre-test as any 
other covariate, or we may look directly at "gain" scores- 
differences between pre- and post-test scores. The major 
advantage of the latter is simplicity and ease of inter- 
pretation. On the other hand, if the relationship between 
pre- and post-test scor.es is complex, the obvious inter- 
pretation may be quite misleading. In calculating the 
gain score we arbitrarily fix the relationship between 
pre- and post-test to be 1.00 — thus a difference of one 
point in the pre-test is fixed to be associated with a 
difference of one point on the post-test. This may not 
always be accurate. For example, suppose that children 
with a pre-test score of 10 end up on the average with a 
^ post-test score of 15, while children with a pre-test of 
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The simplest approach to comparing groups is to. form 
sub-classes by cross-tabulating observations on several 
concomitant variables and calculating pre- and post-test 
means and standard deviations (and possibly other summary 
statistics) for the resulting subgroups. In Chapters III 
and IV f for example, we divided the data into twelve 
subgroups, stratifying on ethnicity, prior preschool 
experience, and entering grade level. Direct comparisons 
were made between corresponding subclasses of the dif- 
ferent groups. Such comparisons will be unbiased with^ 
respect to the variables used in the cross-tabulation. 
While this approach is simple and the resulting statistics 
easily understandable, it generates a mass of information 
which may be difficult to use. Note that the more we 
subdivide our original groins the more control we exercise 
over possible biases, but the fewer observations we have 

f 

per subgroups. Thus the price of greater bias control 
is loss of precision in our estimates. 

We must always face this dilemma unless we are willing 
to assume more structure in the way the covariates affect 
the outcome measure. Our second approach does exactly this 
In using the general linear model or the analysis of 

7 

20 end up on the average with 30. HEre a one point dif- 
ference in the pre-test is associated with a 1.5 point 
difference on the post-test. Using the pre-test "^s a 
covariate where we let the data fix the relationship helps 
us to deal with situations of this kind. 
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covariance (ANCOVA) , we assume that the relationship'' . 
between the outcome measure and the covariate has a 
particular mathematical form. If this assumjalion is 
approximately correct, we can make efficient comparisons 
while controlling simultaneously for many variables. 

The assumiption is that the expected outcome measure 
(dependent variable) value is a linear function of a set 
of independent variables. These independent variables 
may be continuous variables, duiruny variables standing 
for membership in various classif icatory groupings 
(e.g., Ek/El) , or variables representing interactions 
among measured variables or transformations of them. 
Thus we can use ANCOVA to express post-test score as a 
function of variables corresponding to membership in the 
groups we wish to compare as v/ell as a variety of co-- 
variates. It is then possible to calculate the propor-- 
tion of the post-test variance attributable to various 
independent variables, ' In particular, we can estimate 
the variance explained by group membership over and Above ^ 
that explained by the covariates and test its significance. 
We can- also estimate and test the significance of dif- 
ferences between pairs of group means adjusted, for dif- ■ 
ferences in the covariates. Thus if the linear model is 
approximately correct, we have a powerful and flexible 
tool for group comparison. ' 

One probleir with ANCOVA, in addition to possible 
departures from the assumptions of the linear model, is 

Er|c nij>i 



that low reliability .of./ the covariates can introduc© 

/ ' 

bi<^ses into the eatxr/tes and te,sts of group differences. 
But th^ tx^ain difficulty with .AllCOVA Is the fiecessity to 
specxfy the torn oi th© relationship between the oiitcoiie 
variable and the/covarxates* Our, third approach' avoids 

this problem* /' 

/" 

,This approach involves finding pairs of classrooms 
in different groups which arc- close" to the Bwm^ on their 
values of,' a variety of covars^aias. Regardless .of "the re- 

'lationsh'ip between the cavarxates and the outcome iriseasurcr^ 
ajny di'fferctnce botveen tha outcome* scores of the menibars 
of . the pair cannot attributod to differences on the. 

^ covari'ates, xf the r.atchxn-g procedure is .^xact. Thtts 
each" pair provides an unbiai^cd co.^^parxson betwi^t^n 
groups. Smc^ in ptactidi it will, almost never be 
possible to find exact n>atchos, the -pflicriency of the 
matching procedure vmU depend on our ability to find 
♦•good" matcheii. This'' can be a s^^rious pract irul probloi^^ 

If the functional forTn underlyincj tna ANCOVA is 
approximately corrc-ct, it X5 r;uch more efficient than 
matchjing. ^tatchinxj, on th^;^ other hand, h^s the advantage 
of robustr.ei^H ; tnat is, it requires very miramal assump-* 
tions to be vdliu. ' h mnot drawback of t^iktchinq xr. that 
oven though 'it xtr^plicitlY controls for any sort of com- 
plex relationship bctwoon outcont^c? and cov^^riates* it 
gives no information about the nature of these lat ioru^hip 
Finally, t;:orc r,c:riOus prablejt^^ can ari^^ j.n connection 



with unreliabxlxty of the covariates. Hatching on fallible 
covariatos can lead to regression artifacts which distort 
the observctd dafferances between groups* In general the 
larger the differences among covariate m^ans for the groups 
'-m are^ coiTiparin^ and the lower the reliability^, the more 
pronounced wxll be the effect of the regression artifact* 

In sixwMaiy, we will rely on three sets of analyses. 
,The firsts cross-^tabuiation^ has the advantages of ease 
of ~a^ir>terpretatxon and lack of assumptions about the nattare 
of the relationships between the concomitant variables and 
the outcome measures* Its disadvantage stems from a lack 
of precision from Sinaall smpl^ bxzob created. by .subdividing 
original groups on a nu-mber of concomitant variables* 
The second ^procedure^ analysis a£ covariance., gains its 
.strength fjom a set of assujnptions which specify the func- 
tional relationsnips betwaan the concomitant and outcome 
variables. If the assumptions are reasonably accurate 
this ?n€?thod should both reduce biases' and offer far greater 
precision than the first approach* Tho third approach^ 
matching^ again tak^s us off the hook of specifying tho 
functionarrelationsh-ips between concomir.ant and outcome " 
variable*^ but leaves us v/ithoiiit anywhere near the loss of 
precision, of the eross^-'taoulat ion approach* The drawback of 
the* fin.^1 method is that unlike cross-tabulation^ wo do not 
generate: the data to.describo the relationships* 
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Our approach to pre-^enting observed post-test scores, 
"gaiu" score;, and "gain score differences" needs no explan-^^ 
ataon. Some exrianatxcn, however ■> is required for the other 
two seta or analyses. 

vThe analyses us:r.g tno general linear model may be 
divided into two categories. Both use the classroom as 
the unxt of analysis. In the first category are analyses 
Jnj^..js^lxpl<^ regre^^Kion format- with post-test classroom 
aggregates as dependent variables and' aggregate pre-scores , 
child characterin::icK and teacher and site characteristics 
as covarxates. We use this approach in "both the analysis of 
overall differences between PV and Comparison classrooms and 
in contrasting PV models. 

Additionally, in the analysis of overall differences 
between PV and Comparison classrooms we allow the covariates 
to take on different weights for each of the PV and Comparison 
groups. Briefly, we enter all of the covariates with a 
dumjity var-iables standing for memberships in the PV or Compari- 
son group. The covariates are assigned observed values, 
unless there is missing data, in which case the subsequent 
correlations and regressions are calculated on the missing 
data; matrix.* Taken alone,' the resulting equation allows 

ff^ff m using classroom aggreaates is that there is 
very little missing data. Our assumption has to be, of course/ 
tnat tnore IS no oias m the aggregates even though some data 
xs not available for all children in the classroom. 
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for one set of relationships between the covariates and 
the outcome measures within each of the PV and Comparison 
groups. We then enter a new set of the same covariates, 
this time giving them a value of zero if the observation is 
in the comparison group and the observed value of the 
observation if it is in the PV group. This procedure allows 
for different relationships between the outcome measures 
and the covariates for the PV and Comparison groups. This 
may be thought of as accomodating interactions between the 
covariates and the PV' and Comparison groups in their effects^ 
on the outcome variables. 

In the regression analyses contrasting different 
curriculum models we take a somewhat different appiroach. 
Here we create dummy membership, variables for each of the 
models and evaluate the magnitude of the resulting coeffi- 
cients against an overall adjusted comparison group effect. 
In these analyses we only allow for differential relationships 
among the groups on two variables (the PSI pre-test score 
and the proportion of children with a prior preschool experi- 
ence) • Introduction of other interactions proved too 
unwieldy and not worthy of the bother. 

The second set of analyses with the linear model approach 
used a multivariate analysis of a variance framework. This 
allows us to examine a number of dependent variables simul- 
taneously. For the study of differences between PV and Compari 
son classrooms we used a two factor design (models by PV/ 
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Comparison) . Only classrooms in sites with both a PV and 
a Comparison group were included in the design. This gave • 
us an eighteen cell design (9 models by PV/Comparinon) . The 
interpretation of the model to model differences in this 
design is difficult since both the PV and Comparison means • " - 
are pooled to come up with a model effect. However, the.PV/ 
Comparison contrast gives us an overall estimate of the 
differential ef f ectiveneiss of the two groups and . the inter- 
action terms give us some idea of whether there are model 
to model differences in the relationship between the compari- 
son and PV groups. 

When we compare curriculum approaches within the 
analysis of variance framework, two multivariate arjalysis • 
approaches are used. First, we directly compare the PV 
model groups in a one way analysis of covariance format. 
This is a straightforward approach but given the differences 
between models that we pointed out in earlier chapters, it 
might be misleading. Thus, we also carried out a one way 
design with nested PV and Comparison groups withi-n models. 
This let us make one degree of freedom contrasts between PV 
and Comparison groups within a model. Again, only classrooms 
within sites with both PV and Comparison groups were used.* 

*In all of the multivariate analyses we present both univeriate 
and multivariate tests of significance and use a variety of 
child aggregates, teacher characteristics, and site characteristics 
as covariates. The analysis of variance approach is an exact 
least squares solution for unbalanced designs. The particular 
method used calls for estimation of effects by equally weighting 
all appropriate cell means. Covariance adjustments are carried 
^ out around an unweighted mean of the cell means for the covariates. 
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Two problems should be noted with these multivariate 
analyses. First, although we introduce a variety of covar- 
iates, we do not test for homogeneity of the regression 

surfaces. Second, we do not take complete advantage of the 

■ \ 

match between PV and Comparison groups within sites. To 
do this we would have been required to use the site as the 
unit of analysis and carry out a repeated measures design — 
we rejected this for reasons given above. Our only attempts 
to accounte for the match within sites was to eliminate 
from some analyses sites without both PV and Comparison group 
and to include as covariates some site level characteristics 
such as the variable assessing entering elementary grade 
level (El/EK) . . ' 

The third set of analyses used matched PVand Compari- 
son classrooms. As we remarked earlier our purpose was to 
develop an analysis strategy which did not require our 
initially specifying the functional relationship between the 
covariates and the dependent variables and which did not 
entail the loss of precision resulting from cross-tabulation 
techniques. Although we matched at both the individual 
level and in three ways at the classroom level we present 
results from only two of the classroom matches. Results 
from the other matches were highly consistent with those 
reported. We first present the procedures used for matching 
and then consider details of the analysis. Four steps were 
required in creating the matched samples. The steps involved 
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solving a number of theoretical and practical problems. 
Since there are few precedents in the literature we go into ' 
considerable detail both to justify and to explain our - 
admittedly ad hoc procedures. 

1) The first step was to decide upon a set of variables 

to match with. A number of regression analyses carried 

out on both gain scores and post-test scores suggested 

that we use seven background characteristics and the pre- - 

I ■' ■■■■ '. 

test scores themselves as matching variables. The seven 

aggregate background characteristics were mean age in the 

classroom, percent black, percent Mexican-American, mean 

income, mean household size, mean mother's education and 

percent with prior preschool experience. In order to have 

observations for all classrooms we estimated observations for the 

very few missing data points by assigning them the mean for the 

overall group. Three pre-tests were chosen— PSI, Book 3D and Book 

4A. Data were present for all observations for these 

variables. Although the use cf the pre-test scores in 

matching greatly increases the precision of the matching 

it also increases the possibili that regression artifacts 

will influence the estimation o. effects. As a consequence 

we carried out matching procedures on two sets of variables— , 

for the seven background characteristics with the three 

pre-test means and for the seven background characteristics 

alone. Due to the very high estimations of reliability 

for all of our aggregate variables we think that the chance 
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of regression artifacts seriously affecting the estimates 
is small and therefore we favor the ten variable match.- 
Nonetheless, matching with both sets of variables gives 
the reader the opportunity to make up his own mind. 

2) The second step was -to develop a method for simul- 
taneously matching on a number (either seven or ten) of 
variables. Two strategies came to mind. The first required 
ordering the variables in a particular priority and then 
matching classrooms in a step-wise fashion on- these vari- 
ables. Thus we might have first grouped PV and Comparison 
classrooms by categories of preschool experience and then 
within the categories create further subgroups on mean 
mother's education, etc. until all matching variables had 
been exhausted. We rejected this approach, howeve^, for 
two r,easons. We found it difficult to order the variables 
and we found it difficult to create meaningful categories of 
the variables — which due to the aggregation were by „and 
large, continuous. A second strategy, therefore, was adopted. 
In general, this approach required locating each of the PV 
and Comparison classrooms in multi-dimensional space defined 
by the matching variables. Once all of the classrooms are 
located in this space we can then argue that similar class- - 
rdoms are close to each other while quite different classrooms 
are far -apart from each other. Following this logic, we 
6ould then match PV and Comparison classrooms by choosing 
hearby pairs. Actually carrying out this procedure was 



difficult, however, for the ^matching variables are inter- 
correlated. To calculate distances among points in a space 
defined by correlated dimensions requires working with some 

fairly complicated covariance terms something we didn't 

want to do. Calculating the distance, however, between points ' 
in a space defined by uncorrelated or orthogonal dimensions 
is quite straightforward as Pythagorus demonstrated a long 
time ago. We therefore solved our problem by generating a 
number of orthogonal variables to define a subspace within 
the space defined by the original matching variables. The 
technique used for this was principal components analysis. 
All 166 classrooms in the final analysis sample were obser- 
vations in this analysis. --Our procedure was to carry out 
the principal components analysis and to retain for matching 
purposes only components with a latent root greater than one. 
We then calculated scores for each of the classrooms on 
each of the components, retaining the differential weight 
of the size of the latent root. This resulted in five 
component scores for each classroom for the ten variable 
analysis and four scores for the seven variables analysis. 
Within the separate analyses, the sets of scores were uncorrelated 
Moreover, we have some assurance that they are reasonably 
reliable. Aside from the fact that the original observations 
were classroom aggregates and therefore generally of high 
reliability , the component scores can be viewed as probably 
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having greater reliability than the individual variables 
since they are linear composites of a number of highly 
correlated variables. Moreover, the elimination of some of 
the factors with latent roots less than 1.0 may have 
removed some of the random noise from the matching variables 

3. Third, after component scores were calculated for 
each of the classrooms, a distance matrix was constructed. " 
The distance matrix had PV classrooms as one dimension and 
comparison classrooms as the other. Each cell in the 
matrix contained the distance between a PV classrdom and a 
comparison classroom. The distances between classrooms 
were computed by taking the square root of the sum of the 
squared differences between the component scores of the 
classrooms. 

4. Fourth, once we had the distance matrix, we needed 
to find the "best" matches. This is not a trivial problem 
as Rubin (1971) points out. But finding the strategy for 
the best. fit was not the only problem. First, we wanted to 
match not only on the vari bles included in the components 
analysis, but also on the entering grade level of the site. 
Second, we faced the problem of having many more PV class- 
rooms than Comparison classrooms. If*^ we wanted to find a 
match for every PV classroom we would be required to use 
some comparison classrooms two or more times. How were we 
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to deal with dupUcations? Third, we had to decide upon 
some criteria for evaluating the quality of bur matches." 
The first problem was easily resolved — we only matched 
classrooms if they were from sites with the same entering . 
grade level; they we only matched EK PV classes with EK 
comparison classes. The second problem was somewhat more 
complicated. Our resolution of the problem of duplicate 
comparison classrooms was to treat PV models separately. 
The procedure took the entire set of PV classrooms within 
a model and then searched for the "best" match for each 
classroom from the entire set of comparison classrooms. 
No duplications were allowed within PV models. The idea 
was to not constrain the number of degrees of freedom for con- 
trasts within models. This approach essentially created eleven 
separate sub-experiments, each comprised of PV classrooms 
within a model matched with comparison classrooms from the' 
entire pool of comparison classrooms. Since there were at 
most twelve PV classrooms within a model and 65 comparison 
classrooms, we had a lot of leeway in our matching to accomo- 
date extreme PV classrooms. 

Third, we chose a least squares criterion for evaluating 
alternative matches. Our argument was based on the fact 
that we were matching the PV clc.ssrooms within a model 
altogether rather than independently — since we did not 
allow duplicates within models. , We therefore needed an 
overall measure of the average differences among different 
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combinations of matched PV' and comparison classrooms in 
order to get some idea of the best combination for models. 
We chose the criteria to be the minimum value of the sum 
of the squared instances between the matched PV and compar- 
ison classrooms. Another possibility was to choose the 
minimum sum of the distances between matched PV and compari- 
son classrooms. In practice the two seem to result in 
essentially equivalent matches. With all these decisions 
made, we only needed to find the "best" matches. We did 
not solve the problem — like Rubin, we settled on heuristic 
devices.* We used four general strategies. 

In each of the following steps we deal with the models 
separately. The first step in each strategy was to select 
for each PV classroom in a model the 12 closest comparison 
classrooms. We called this a "reduced" distance matrix. 
If there was no overlap in the closest matches we were all 

• set ~ we simply chose the closest ones. If however , there 
were comparison classrooms that were closest to more than 
one PV classroom, we had to figure out some way of selecting 

' the best combination of matches. One approach started by 

taking the shortcut distance between any of the PV and compari- 
son classrooms and accepting that as one matched pair of 
classrooms. Since we did not allow duplicate comparison 
classrooms within a model, we then had to eliminate from the 
reduced distance matrix all occurences of the matched compari- 
son classroom. After that step was carried out, we again 

IC . 

^ *For those of you wh& think this is a $ifl\pjJLjp^^sk, we 
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selected the closest match etc. for all of the PV classrooms. 
Once we had ipatched each of the PV classrooms with a compari- 
son classroom, we then computed a sum of the squared distances. 
The second approach used was to select the PV classroom 
that had the worst match in terms of distance with any of 
the comparison classrooms. This PV classroom was matched 
with its nearest comparison classroom, the comparison class- 
room was eliminated from the reduced distance matrix and the 
process was repeated for the PV classroom with the next worst 
match. A sum of squared distances was then computed for this 
procedure. We might call the first procedure a heuristic 
maximin procedure and the second a minimax procedure. 

The third approach was to select a PV classroom randomly 
and match it with its closest comparison classroom. Then 
the comparison . classroom would be eliminated from the reduced 
distance matrix and ^afiother random PV classroom chosen, etc. 
A sum of squared distance was then calculated for this 
procedure. The fourth procedure took the best result from 
the other procedures and tried out a limited power approach 
to see whether the overall sum of squares could be reduced. 

In general, the power procedure slightly improved 
upon other procedures. We might note that there were con- 
siderable differences in the sums of squares of the distances 
for the four procedures. Within each model, then, a heuristically 
"besH:'-- matched set of PV and Comparison classrooms were chosen. 
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This procedure was carried out independently twice — 
for the 10 variable, five component solution and for the 7 
variable, 4 component solution. Additionally, for each 
solution, the matching procedure was carried out separately 
for the sample of all Level II and III si-tes and for the 
sample of only Level III sites. We had to' cairry out the 
Level III only matches to insure that we could succe-ssf ully 
analyze the Stanford-Binet . 

To analyze the data we decided upon a one way nested 
analysis of variance with one covariate. Our procedure 
treated each of the sites as a level in a one way design 
using the difference between the matched PV and comparison 
post-test classroom means as the depeijdent variable and 
the difference between the matched PV and comparison pre- 
test means as the covariate. Correction for the reliability 
of the covariate were carried out using the Lord-Porter 
(see Porter, 19 72) technique. Because we knew a priori that 
the grand mean for the covariate should La zero (since it 
is a difference between pre-score means for matched classrooms) 
we calculated the covariance adjustment around a zero mean rather 
than around an observed grand mean. Overall PV/Compa.rison 
contrasts and model effects were calculated by pooling 
unweighted adjusted means across- the sites. 

All of this sounds pretty complicated for a simple one 
way analysis of variance with covariance adjustment* Un- 
fortunately, little theory and thought have been given to 
Q the .practical problems of dealing with matching in quasi- 
fima^ta experiments of this sort and^^^|a^ l^onsequence many of our 
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pt^ocedures seem more than a little ad hoc. Yet for a 

nu^ex reasons it seems to us that this procedure outlined 

above might confecibute a lot of power to our analyses. 

First, it allows us to deal with two y^ty practical 
analysis problems. As 'we pointed out in Chapter 11^ W0 
have no comparison classrooi^s for two models the 'En^lers 
and REC* Moreover , for two other sit^^s we lacked either 
on-site or off-site comparison classrooms « Since direct 
comparisons among models seems to be a weak approach 
because the sites within f^odels seem to differ on soto 
important characteristics — we have t^^-ndad to place oar 
reliance on an indirect comparicon- as^ong models* mc^diated 
through" the contrast be-tween models and their eo??i*parison 
classrooms* But to carry out this procedure we need s&m 
assurance that tJie PV and comparison classrocnn^ are soff^- 
what equivalent* Pairing by location does this for thosa 
sites with both PV and comparison classroor^^s hnt it does 
nothing" for the Enablers^ REC-fM the t^o other sites ^-/ithoyt 
comparison classrooms. Only a r.atehing strategy could alio^^* 
us to place these problem sites m an analysis contrasting , . 
■ .models with comparison ciassrDO?!?s . Second .t even for sites 
which have both PV and comparison classrooms, certairn ptdbl^tp■B 
exist in the analysis. As we noted^ without the site a^s 
^ the unit of analysis^ there is' no natural way to use the 

pairing by locatiorsi to reduce the error term in our analysis'; 
classrooms are, not matched within sites and often there are ' 

FRir ^^'^ than comparison classroofe. Matching classrooms by 

- ■ . ■ .. J « 4 (| 

w ratheg than bv location .eliminatea each .of theaa ; ^ 
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probleros. The roatchea classrooms pairs^ can be treated as 
the unit of analysis m;d the design becomes balanced with 
regard to, eorrparisons withm models. 
^ ■ Third* as we'notad earlier, the matching procedure 

as a strategy for control does not require us to specify ' 
the functional relationship between the control variables 
and .-he outcome variables as other control procedures, 
r&iymg corlipletely on the general linear model. This stHiKes 
us as an e5<treiBely important argument granting, of course, 
that we have chosen the right variables to uvatch with. 

One final reraark. Two principal problems that analysts 
have raised about matching st&n from issues of the reliabi- 
lity of the^ovariates and the similarity of the matching 
covariate^v i... the two sarnples m their distributed charac- 
teristics. By and large, we think the variables used for 
snatching are eKtreir^ely reliable, and by and large, the 
characteristics of the co^'ariates in the samples being 
jsatched are very sij^iiar fst?e Cnaptors II and II IK Yet 
It still ue(;->s appro^iiate to vatch out for extreme cases ;n 
our analys*;:-, of th<.; f?'.atehc-d sar.pic?ii = 

In th« ncKt two chaptera use the procedures outlined 
dbovv, eCh.jpt,er v; cv.ni,iderte tf;- question of whether there arc 
overran dufc-r-r.ccr., ^r. erfecta. between PV and Comparisen class- 
roi>-ri, Chapt.?r Vll i-ocaii.es o;. tju? question of model to r»odel 
dii'i-:-s once; t ivenersG 



Chapter VI ■ 

o\i:rall di ff erences in the effects of PV 
and ' omparison classrooms 

Introduction : 

T he eleven preschool models in the- Planned 
variacxon stuay nave somewnac differing eropnases on the ^ 
outcomes raeas' red xn this study. We might therefore ' 
expect to find outcome differences among Planned Variation 

r 

classrooms. But this does not imply that we would expect 
an outcojnc averaged across all PV classrooms to be signifi- 
cantly different from an average of all Comparison classroons. 
Because the expected differences among models are lost 
in looking at overall averages, it is difficult to attach niuch 
substantive meaning to a contrast of all PV classes versus 
all Comparison classes. If the degree of curriculum emphases 
in a measured domain does affect the outcome, then a pre- 
diction that the average of all PV classes would show more 
change than the average of all Comparison classes requires 
the assumption that the modal emphasis in this domain is 
greater for the PV classrooms. We have no way of obtaining 
this inforr.ation and thus no way of knowing whether to 
expect PV classrooms on the average to "do better" or "do 
worse" on our itKiasures tiian the Comparison classrooms. 

The main reason for contrasting the overall effects of 
PV and Comparison classes is to determine whether the effect 
of the extra $350,00 per child spent on children in PV class- 
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rooms has an effect on measured outcomes. For while we 
cannot identify modal curriculum emphases for the two groups, 
we can speculate that the additional personnel and materials 
available to the PV classrooms might have an effect. 

This chapter reports a series of analyses on the differ- 
ential effects of PV and Comparison ctasses-*^^-^^^ has 
four sections. In the first section we contrast raw "gains"" 
for the total PV and Comparison groups and for 12 subgroups 
within each. The child is the unit of analysis. The 
purpose of these contrasts is to give the reader some feel 
for observed differences before we carry out -procedures 
of control and adjustment. Section II reports on a series 
of regression analyses which have a PV/Comparison dummy 
variable and a set of background and teacher characteristics 
(with separate slopes for PV and Comparison groups if necessary) 
as independent variables. In Section III, we report on 
a series of two-way analyses of variance. The approach used 
is a multivariate exact least squares solution with models 
as one factor and PV/Comparison as the second factor. In 
Section IV, we report tv;o sets of one-way nested analyses 
of covariance using matched samples with adjustments for 
fallible covariates . 

Each of the reported analyses offers some slightly 
different information about the effects of PV \ 
and Comparison classes. As a consequence, there are slight - 

i 
! 
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differences in the estimates of effects. The 
general conclusion that can be reached from all of thg»gp> 
analyses is that there are no differences between the PV 
and comparison groups in effects on the measured outcomes . 
This was -not an unexpected fi^nding and in no way implies 
that Planned Variations is ^a failure. For at no time was 
an objective of the Planried Variation study to demonstrate 
that the simple infusion of funds into preschools would have ' 
an effect. Rather the intent of the Head Start Planned 
Variation study has been to invest igate differences in the 
processes and outcomes of different preschool curriculum 
models.. To do this^ the Planned Variation strateg y required 
that preschool curricula be selected and studied for a variet y 
of reasons — not solely because they all intended to maximize 
outcomes on the variables we have measured. 

' I . Differences, between the PV and Comparison samples — 
Observed overall subgroup^ changes . 

In Table VI-1 we present some overall descriptive stat- 
istics for the; PV and Comparison groups. As discussed in 
earlier chapters , there is considerable similarity between 
the PV and Comparison groups on pre and post- test means and 
variances. The only test which looks very/different for the 
two groups is the Stanford-Bine t. Here we see that PV 
children , on. the average , increase their /Bine t scores ixy 
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roughly six points while Comparison children increase their 
Binet scores by only three points. This difference of three 
points is roughly 20-25% of the standard deviation of the 
Binet for these groups and roughly 18-20% of the standard' 
deviation for the Binet for the nation as a whole • None of 
the other differences in overall observed gains exceeds 
10-12% of the standard deviation for its test. 

Table yi-2 shows tests of significance for the differences 
in overall mean gain (at the bottom of the Table) and for 12 
subgroups of chiid-r^ . * Three overall differences in "observed 
gains" are statistically significant at the ,05 level or 
beyond. Two pf the significant differences favor the PV 
group (Book 4A and the Stanf ord-Binet) and one favors the 
comparison group (PSI) . However, neither^^the Book 4A or 
the PSI difference is of sufficient magnitude to be of 
great interest — in neither case does the difference exceed 
one- tenth of the post- test standard deviation. 

In the body of Table VI-2 we observe nine statistically 
significant differences out of forty-nine possible. For 
three of the grotJps two tests show statistically significant 
differences. Mexl<3an American children without preschool 

*The figures in Table VI-2 are differences of difference 
scores. The computation of the scores had two steps. First, 
the pretest mean for the PV group was subtracted from the post- 
test mean for the PV group giving us the PV mean "observed 
gains". Then the pre- test mean for the comparison group was 
s-ubtr acted from -the post-test -mean for the comparison- group 
giving us the comparison group mean "observed gain". The 
comparison "observed gain" was then subtracted from the ' PV 
Q "observed gain", giving the differences presented in Table VI-2. 
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experience in El sites appear to gain more in the PV classe's 
on the Book 4A test and on the Motor Inhibition test. Black 
children without prior preschool iji El sites appear to do 
better in Comparison classes than in PV classes on the Motor 
Inhibition and the Stanford-Binet tests • And black 
children without preschool experience in EK sites tend to 
gain far more on the Stan ford-Binet if they are in PV 
classes but they tend to gain less on the PSI if they are in 
PV classes. The other three significant differences are 
scattered among the remaining seven groups. 

There appears to be only one consistent pattern in this 
table. There is a modest tendency for PV children with prior 
preschool experience to do somewhat better relative to their 
Comparison groups than PV children without prior preschool exp- 
erience do relative to their Comparison group. This holds for 
all tests but the Stanf ord-Binet • Perhaps Hea d Start programs 
with systematically planned curricula are more effective 
for second year preschool students relative to conventional 
Head Start curricula/ than they are for first year preschool 
students. 

The overall comparisons at the bottom of Table VI-2 are con- 
trolled only for the pre-test (and assume a perfect relationship 
between pre and post-test.) The contrasts in the body of 
Table VI-2 control physically for ethnicity, prior preschool 

i" 
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experience, entering grade and their interactions as well 
as for the pre-test (again assuming a perfect relationship 
between pre and post test) . When we contrasted the PV and 
Comparison gains controlling only for the pre-test, we 
found statistically significant differences on three of the 
five variables, two favoring the PV group • Yet when we 
look more closely at the data and introduce the three control 
variables we find only 9 of the 49 contrasts statisticallv sia 
nificant with 6 of the 9 favoring the Comparison qrpup. 
The essential message here is that the introduction of control 
tends both to reduce the proportion of statistically 
significant findings and to cloud the question of whether the 
PV or the Comparison children are, on the average, gaining 
more. This suggests that observed differences between the PV 
and comparison groups may be due more to initial and controll- 
able differences^ between the composition of the two groups 
than to the effects of their Head Start experiences. In the 
following sections v/e pursue this issue. 

II.. Some regression analyses v/ith a PV/Comparison group 
membership variable, and a number of covariates . 

The issue addressed is whether there are stat- 
stically significant PV/Comparison group differences which 
express themselves in a general linear model framework with 
the classroom as the unit of analysis. The approach 
is straightforward. In multiple regression terms, we 

' 1 4 5 



-138- 



examifte the coefficient for a dummy variable (indicating 

membership in either a PV or a Comparison c\ass) which is 

\ 

entered in a regression equation v;ith a number of control 
variables assessing classroom aggregate characteristics of 
children, teacher and site characteristics and wi\h post- 

tpsf scores _as the dependent variable. The two gro'Ups are 

allowed to have separate coefficients for each of the dummy 
variables.* 

Another perhaps simpler way of looking at this analysis 
is to think of it as a two group analysis of covariance — - 
in this instance the two groups are the PV and Comparison 
groups and the covariates are. the "control" variables listed 
in the footnote below. 



*For the PSI , Book 3D and Book 4A we present results from 2 
analysed. In analysis 1 on the total sample of classrooms 
ve use PSI pre, Book 3D pre, Book4A pre, percent female, 
percent prior preschool, mean age, mean income, mean mother's 
education, mean household size , percent Mexican American, per- 
cent Black, years teacher experience in Head Start, teacher 
race, teacher aide years in Head Start, teacher certification, 
average staff working conditions, and whether the site is El 
or EK as control variables. In analysis 2 we limit the samole 
to the Level III sites and use the Stanf ord-Binet pre-score as 
an additional control variable. V7e also use the Stanf ord-Binet 
post-score as a dependent variable in this set of analyses. 
For the Motor Inhibition post-tes't we limit the sample to the 
classes with valid pre and post Motor Inhibition and use the 
PSI, Book 3D, Book 4A and Motor Inhibition pretests as 
well as the other child aggregate, teacher and site character- 
istics as control variables. In all of the analyses we allov; 
for separate slopes for the PV and comparison groups. Following 
Cohen (1971) our procedure for doing" this was to calculate tv70 
sets of control variables (or covariates) . The first set have 
observed values for both the PV and comparison groups. The 

second set are assigned a value of zero of J:he cj^^ 

a comparison classroom and the observed value if the unit is 
a PV classroom. The first set of covariates are forced into the 
equation and we then let as many of the second set (which 
assess differential slopes) of covariates enter as possible. 
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Three separate sets of analyses were carried out. In 
analysis set 1 the dependent variables were the PSI, Book 3D, 
and Book 4A. The total sample of classrooms was used for this 
analysis. Analysis 2 utilizes only classrooms in the Level III 
3ites. The dependent variables were the Stanf ord-Binet , the 
PSI , Book 3D and Book 4A. Analysis 3 was conducted on the sample 
of classrooms with valid Motor Inhibition pre and post-test 
scores. The Motor Inhibition test was the only dependent vari- 
able for analysis 3 . 

Table VI-3 gives pre and post-test N's, means and standard 
deviations for the five tests used in the analyses. Data from 
these analyses are shown in Table VI-4. In columns 1 and 2 are 
zero-order correlations of the dummy PV/Comparison group mem- 
bership variable with pre- and post-test scores respectively. 
None of the correlations is statistically significant and none 
of the differences between the pre and post-test correlations is 
Ifegnif icantly different from zero — though the post-test cor- 
relation for the Stanford-Binet approaches statistical signifi- 
cance as does the difference between the pre and post-test 
correlations for the Stanf ord-Binet . 

Column 3 contains the standardized regression coefficients 
for the group membership dummy vai'iable for the total equations 
— allowing for separate coefficients on the covariates for the 
two groups. Column 4 contains the same group membership standard- 
ized coefficients for an equation allowing for no group by co- 
variate interactions (i.e. only one slope for each covariate is 
allowed.) In no instance does the group membership coefficient 
reach statistical significance. Clearly the PV/Comparison member- 



• ship variable has little predictive power in these equations. 
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Kotos to Tabl& Vl-4 



+ PV is coded i, comparison is coded 0. 

+ All cox'ariates wore entered as though the rearession • 

planes were entirely parallel. Then as many FV covari^jte* 
as necessary were entered to adjust for differences in 
slopes. Stepping v/as tcrRinated ^hen the standard error 
Of the equation reached its ioi.^©st point o 

T- For analysis i the covariat€?s were PSI, Book 3D and Book 4A 
pretests, percent female, percent prior. preschool , nm&n 
age,- me&n income, mean mother *s education, wean household 
si2e, percent Mexican hmetican, percent black, vears teach^- 
experience In Mead. Start, teacher certification*, averaae" " 
• working conditions, teacher aide vears of 0.xp»»rier-o 

and whether ^tht site is an El or EK siti. For analysis i-" 
. tne Scanfora-dinot pre- test was an additional covariate, ^ 
J-or analysis 3 the Motor Inhibition test was an addit*ona» 
cova.jriate — an analysis 3 the Stanford-Bmet pre-test vas 
not used. 



"14 3 = 



Thn sUght di:-.r.nce. between tlw standardized coefficients 
for th.. entir. an^i tcr the eqaat.ons allowing for only 

one overall for e.c;. covarxate reflects th« fact'^ 

that ,n .e^., .a.at;.n. . f.« of the 17 or so covarxates h^ve 

•■-.-I-.... tc. t.-H? PV group than iG th*.- 

cof|.ari..an : ..ner.l. h4.wcv.-r . ^ the provision for 

«.ffer,-r;t. „ ^ ^^^^^^ explaxned 

varxano., ov«. c„,:acca equations vn.ch provicte for only one 

i^lope^ Horc,;.v..r, tr,.. vazi.bles which appear to operate diff» 
' > M''n»;r,a..y w..re not the samt frcjn ea- 

VN... in.dic..t..;;.. along w^th very rlight 
eh.n^e th. r..ai..ta.::. ..f t.h. f-/C..^.« x^on coefficient, that 
th. r.l.,,o..,.p t.i.evr. cov.r,.te3 and an outeor^c ..easure 

«^ ^..m^raiiv Co.^parison group., m 

otner wata.. . important mter^ 

action. th,.::., cc,v.r..t.:.r ..... t:;c PV/Compar isun group v.r.a.>lo.- 

j^-j,^:.T .;r;..r""'' ^ ^-^^lu^tion did an ©r even 

■ .j!.* Vjr ■a-.t.'n ^-itt'^'r t^'^-- — r-f-^ 



» 
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- ^ j-l ^'r- ^ ^.''^ • ...... ^ .... 
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Column 5 xn Table shows the percentages of variation ex- 

plainad by the total equations. The percentages range froji^ 42.2% 
. for th€ Motor Inhxtlitxon to slightly over 80% for' the PSI 
in analysxs 2. In all instances the equations are highly 
significant and indicate that while the simple linear model 
does not explain all of the variation ^ it does very well 
vAosX instances* 

Three conclusions can be reached from|/this section. 
SlXgl^j^fegj^^d^e^^l xng with the classroom as the unit of analy- 
gJu£....±Sl^Li^ sample, th ere are no statistically 

•tiaiLLil£gM^^^lg^j^^^^^ the PV and Comparison 

gLrouil^...either without cont rols or after extensive linear 
£2£ll£SLl§^9X., ^y^^of. the five post-test scores, 
^SSSIl!h.^.±L..J^:^..^ the PV/Comparison 

£gM£-lSl^At_io<gjiS^a if fitting an .equatidn without con- 

gj^XlL^ slo pes f or ^he two groups is 

i^^.M^..MllJ£A<mk.^^ for sep arate slop< 

^^A^Aimk^} . ior thc^ various covari ates. Third, fdr at least 
!J:!£g:.^.£L s C PS I , Book 3fi,..^ggff^B 

Jtj::...Jj:I:.:L equati o ns account for 
i-'^3k'''^l^^l:!X..^^^^ va nation 

i^^l^^^-^J2l^l^S..Jl^ analyses) . 

Stan t or d 




dcalin^l wit;, n Canaiy:i:,s 11 cr .H ianaly;.is 2 and tor tho MI) 
^ovariatcH ^hc^*:? i.et?m 1 iki,^ few ;;-,tcract ions . Thougn the bverali 



^C^t IS somewhat better with ;>or.c* of the separate slope coefficients 
wffljddod^ the overall gam in precision seems slight. ' '-^ \ 42 
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IV, Results from some Exact Least Squares Solutions of 
Unbalanced Two Way Analyses of Covariance 

This section reports statistics from three exact least 
squares solutions of Unbalanced Two Factor analyses of 
covariance. The two factors are Models (9 levels) and 
PV/Comparison classes • The samples include only classes 
from sites with both PV and Comparison classes. 
Classes are pooled across sites into models. Only nine 
models are represented--REC and the Enablers are left 
out of the analyses. Data from the three analyses are 
presented in Table VI-S. 

For analysis 1 the dependent variables were the PSI, 
Book 3D and Book 4A. Covariates are listed in Table VI-6 
and with one exception, are the same as those used in the 
regression analyses reported in the preceding section of 
this chapter. The first two columns show the overall PV 
and comparison group N's — the number of classrooms used 
in the analysis. Columns 3 and 4 show the estimated 
combined means for the PV group as a whole and for the 
Comparison group as a whole. These means can be interprel:ed 
as the unweighted average of the nine adjusted cell means 
for the levels of the PV/Comparison factor. Column 5 
shows the estimated effect — the difference between the two 
combined moans. The adjustments are calculated around 
unweighted means of the covariates, 

A co.Tiparison of the adjusted means in Table VI-S and 
the raw means presented m Table VI-1 shows a strong 
ERXC similarity even though the samples were slightly different 
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(the sample used in Table VI-6 is smaller due to 
the eliminatioii. of classrooms in sites without a 
comparison group) , eWR though the means in Table VI-6 
were unweighted avefageit;'J^:S^S^^^^t^ means in Table Vl-1 
were v/eighted averages of cVethm^l&tmf tx^ 
even though one set of means was adju^l^l^' li^tt^^ 
Others weire not. The magnitude of means for the 
py and comparison groups seem remarkably stable even 
jgiven changes in samples, methods of estimation and 
methods of adjustment. 

Of the three estimated differences for analysis 
1 only the PV/Comparison contrast for Book 3D shows 
statistically significant results. The difference 
(-.470), favoring the comparison group, is roughly 
0.15 of the standard deviation of the individual 
post Book 3D test and is significant at the 0.05 
levels However, since the PV/Comparison effect is 
correlated withTthe model to model effects, the PV/ 
Comparison effect does not reach significance when 
\ the model to model differences are taken out first 
(see the F test for PV/Comparison group differences). 
MorepR^f^^%h^^verall multivariate test for differences 
between the PV and c^garison mean vectors is not 
stat?i^tically significant. This indicates that the 
Book ^ effect is marginal at best. For neither of the 
Other two variables does an ^estimated difference" reach 
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even 10% of the post-test standard deviation. 

The last two columns of this table indicate the 
overall tanivariate F tests for the interaction term and 
for model to model differences. In no case was the uni- 
variate F for interactions statistically significant 
though the multivariate test for interactions did reach 
statistical significance — 'P<:.05. This indicates first, 

that we are generally justified in interpreting main 
effects, and second, that there is a strong correspondence 
between the adjusted means for the PV and comparison groups 
within models as well as overall. The significant multi- 
variate F for interactiiihs , of course, tempers this final 
conclusion. 

The last column indicates that for the PSI there are 
strong model to model differences in adjusted means; of 
course, these means are calculated by pooling both PV 
and comparison group classrooms and, therefore, interpreta- 
tion is difficult. The univariate F's for model to model 
differences for the other two variables are not statistically 
significant. The overall multivariate F for model to model 
differences is highly significant. 

Analysis 2 used a sample of only classrooms in the Level 
III tested sites, again eliminating those classrooms in sites 
without comparison groups. The reduction of the sample to 
only Level III sites allows us to include the Stanford 
Binet in the analyses — the post^-test is included as a 
dependent variable and the pre-test as a covariate. Other 

ERiC ^ .) 5 
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than the sample reduction and the addition of the Stanford- 
Binet analysis 2 is the same as analysis 1. 

Analysis 2 adds little information about t^he PSI, 
Book 3D and Book 4A except to indicate that the addition 
of the pre-test Binet as a covariate and the change in 
.the sample results in a non-significant estimated differ- 
ence in the PV/cdmparison contrast for the Book 3D 
test. The magnitude of the Book 3D difference (now 
-0.366), however, changed only slightly from the previous 
analysis. 

The largest change in differences can be seen for Book 
4A — it goes from an estimated difference of 0.332 
in analysis 1 to an estimated difference of -0.018 in analysis 2 
neither effect is statistically significant. Once 
again the univariate interaction effects are all 
insignificant. In contrast to analysis 1, however, the 
Book 3D test as well as the PSI showed statistically 
significant differences among models. The Stanf ord-Binet 
also showed statistically significant model to model 
differences. 

In analysis 3, four post-tests were included as dependent 
variables: PSI, Book 3D, Book 4A and the Motor Inhibition. 
Table VI-6- ^hows results only for the Motor Inhibition test. 
For the Motor Inhibition there are no significant differences fc 
either the PV/Comparison or the Univariate Interaction ^contrast . 
The univariate test of model to model differences is statisti- 
cally significant at the .01 level. 
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By and large these findings are consistent with the 
findings in earlier sections of this chapter. There is 
little indication of btatistically significant PV/c6mp- 
arison group differences^ The only exception, to this is 
the small statistically significant effect found for the 
Book 3D test on analysis 1« 
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TABLE VI -6 

N's, Estimated Combined Means and Estimated Effects for 
a PV/Comparison 1 degree of freedom contrast. Design is 
a crossed two way analysis of covariance; nine models by 
PV/Comparison. Tests of significance for the estimated 
effects are shown in the Table. Tests for significance 
of PV/Cbmparison by model interaction and for overall 
model to model differences are also shown — note that 
model to model differences pool PV and Comparison groups 
together. Only sites with both PV and Comparison groups 
are included in the analyses. The classroom is the unit 
of analysis. See footnote for the listing of the covar- 
iates.^ 
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+The covariates for Analysis 1 are PSI Pre-test, Book 3D 
prc-test, Book 4A pre-test, mean age, %' Black, % Mexican- 
American, % female, % prior preschool, mean income, mean 
household size, mean raoLher*s education, teacher experience 
in Head Start, teacher certification, average staff wojjrk- 
ing conditions , experience of teacher aide in HS and a 
dummy variable for El/EK. For Analysis 2 all of the same 
covariates were used and the Stanf ord-Binet pre-test was 
added. For Analysis 3 the same covariates as Analysis 1 
were used with the addition of the Motor Inhibition pre-test. 

-f-f ' 

Notes on Multivariate F-Tests. 

1. In all instances the multivariates F for models 

were statistically significant beyond the •001 level, 

2. In no instance was the multivariate F for the PV/ 
Comparison contrast statistically significant. 

3. In all instances the multivariate F for interaction 
was statistically sigpificant .03 < p <.05, 
though in no instance' was a univariate F significant 



In analysis 1 the estimated effect for the PV/Compi. son 
contrast was statistically significant at the .05 level 
favoring the comparison group. Since, however, this effect 
is correlated with model effects and model effects were 
removed before it; was the F-test for significance, was less 
than the value required for statistical significance at 
the .05 level. It was significant at the p < .08 level. 

* = statistically significant at the .05 level 
** = statistically significant at the .01 level. 
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V. PV/Comparison group differences using Matched Samples 

In Chapter V we described the procedure used to match 
Comparison classrooms with PV classrooms. Here we present 
data from two sets" of analyses each on two matched samples 
of classrooms. In the first sample^ matching was carried 
out on seven background and three pre-test variables. 
In the second sample matching was carried out using 
only the seven background characteristics (see Chapter 
'V for a description of the variables). 

The first set of analyses for each sample was carried 
out on the total final analysis sample of PV classrooms 
and their matched Comparison classrooms. Four dependent 
variables were used in this analysis; PSI post-test, 
Book 3D post-test, Book 4A post-test and Motor Inhibition 
post--test. The dependent variables are calculated by 
subtracting the Comparison classroom mean from its 
matched PV classroom mean. For the PSI, Book 3D and Book 
4A, the sample is 101 matched classrooms divided 
among 26 sites. For the Motor Inhibition test, due to 
missing data, we have only 75 matched classrooms in 23 
sites for the first sample and 76 classrooms in 25 sites 
for the second sample.* The second set of analyses were 
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*The matching was carried out without regard to whether the 
Motor Inhibition test had a valid pre- and post-score for 
both the PV and matched comparison classrooms. Thus the- 
usable nun-iber of classrooms is somewhat loss than the total 
possible number of PV classrooms with a valid pre- and post- 
Motor Inhibition score. 
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carried out only on the Level III sites. The dependent 
variables were the Book 3D, Book 4A and Stanford-Binet 
mean difference scores. For Stanford-Binet this gave us 
61 classrooms in 16 sites and for the other variables 
we have 62 classrooms in 16 sites for this analysis. 

For all analyses one covariate was used — the covariate' 
was the pre-test score for the particular dependent vari- 
able being used. The covariate was calculated by subtract- 
ing the comparison classroom pre-test mean from its matched 
PV classroom mean. Furthermore, In each analysis three 
different levels of estimated reliability of the covariate 
were "corrected for" (1.00, 0.80 and 0.60). The rationale 
for "correcting" for the reliability of the covariate 
here and not i:i other analyses was that the procedure of 
taking a difference score of matched pairs of classrooms 
producer covariates v;hich are substantially less 
reliable than the original aggregated means.* 



*The procedure used to adjust the covariates for unreliability 
was the Lord-Porter (Porter, 1972) formula. Though this pro- 
cedure produces the correct effect estimates it probably does 
not produce the correct standard error — it is probably a con- 
servative estimate . By and large, however, we are less concerned 
with statistical significance than with the estimation of effects. 
We can estimate the reliability of a difference score using the 
follov;ing formula: 

" ^ ^^^^ ^ reliability of the difference 

where r^^= reliability of PV scores 

^ ^ab^ ^bb"^ reliability of Comp. scores 

r^)3= correlation between matched 
PV and comparison classes 

An example of this for the PSI in analysis 1 for the first 
sample we have: 

ro = ' - '^-'^^^ = 0.8314 

O . 2(1 - .822) 
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Table VI-7 presents data from the analyses on the two 
samples. Consider analysis ,1 first. Here we have matched 
on the seven background variables and the three pre-test 
variables. Columns 1 and 2 show the classrooms and sites 
in the analyses. Columns 3 and 4 show the observed matched 
difference scores for the covariate (the mean difference 
between the comparison and matched PV classroom means 
ignoring the sites) . By and large these differences— when 
compared with the standard deviations for the pre-tests 
taking the individual or the classroom as the unit of 
analysis — are small (see Tables VI-1 and VI-3) . Only 
for the Stanford-Binet is there a difference in matched 
pre-test scores exceeding 0.10 standard deviations of 
the individual pre-test scores. 



Given the correlations between the matched PV and Comparison 
post-tests shown in the last column of Table VI-7 and" the 
estimated reliabilities given in Chpater II and Chapter V we 
estimate that all of the reliabilities of the covariates lie 
in the range of 0.60 to 1.0. Thus we have used three estimates 
(0.60, 0.80 and 1.00) of the reliability of the tests--in order 
to obtain some idea of the impact of the correction procedures. 
We should note that this approach to correcting for the reliability 
of the covariate (in- addition to probably overestimating the stand- 
ard error) ignores the critical problem of choosing an approoriate 
original reliability estimate^ — should v;e choose an internal relia- 
bility estimate, a test-retest estimate over what period of time, o 
a parallel forms reliability estimate again over what period of 
time? Our reason for ignoring the issue is that we have only one- 
estimate of tne reliability of the tests--an internal KR-20 
estimate. Though we might have adopted Campbell and Erlenbacherj; 
approach of adjusting the reliability until the coefficient of 
the covariate was 1.0 tnis seemed inappropriate if we also r-re- 
sent tlie overall "gam" scores--since this was all the procedure 
supplies UG with. 

As we point out in Chapter V the adjustment for the cov.ir- 
iatc takes place within each site around a covariate mean of 

The estimated coefficient is taken fror, 
the. pooled within roaression of the dependent variable on tho 
■ covariate . 

ERIC " 
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Moreover, as we might have exf^ected from previous 
analyses the mean differences between the PV and matched 
comparison post- test classroom means are not great, 
.indicating that there is little difference in the^ effects 
; of PV and comparison classrooms overall. Columns 5, 6, 

I and 7 show the estimated differences between the PV and 

1 

' Comparison groups after covariance adjustment. Differences 
are shown for three levels of reliability {1.0^ 0.80 and 
0.60). The estimated mean differences were arrived at 
by pooling the unweighted adjusted means of the sites 
across all of the sites. Columns 8, 9 and 10 show the 
standard errors for thej estimated differences.* Only one 
test in the sample 1 analyses, the Book 3D te^t in 
analysis^ 2, (Level III sites only) reaches statistical 
significance. The adju^stment for the reliability of the 
covariate appears to do little to this estimate^,-! t ranges 
from -0.4674 to -0.4950 favoring the Comparison ^roup. 
This is a similar finding to that reported from the multi- 
variate analysis of variance. In both instances- we find 
one of the estimates of the Book 3D differences to be 
statistically significant at the 0.05 level, favoring the 
Comparison group, with a magnitude of roughly 0.50 points 
or one-sixth of the post-test standard deviation for 
individual children. 
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*The number of degrees of freedom for the estimates -are equal 
to. 1 and N-k--l where N is equal to the nujndber of classrooms, 
and K is equal to the number of sites. Thus for- analysis 1 
using the PSI the number of degrees of freedom are 1 and 74'-- 
to be statistically significant beyond^he 0*05, level the 
ratio of the difference to the standard error has to be 
greater than 1.99* MM u H 
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Column li shews an F statistic for the test of honjogenexty 

regression slopes within the sj.fes m the analysis* For • 

otkm of thB analyses using sample 1 the F is statistically 

significant at tha 0*05 level indicating that the within 

coefficients for the separate sites are statistically 

different, from ona another # and therefore # that ad3ust:m^nt 

procedures may ba inappropriate. Finally coliium 12 

shows the correlation between the isatched classroo^is on 

t 

the particular pre-test---^thus # for the PSl m analysis I 
the fffiatching produced a correlation of 0.1^81 between the 
PV and Comparison claEsrooias, 

The data for the second patched sa.?npl© are presen^ted 
in the second half of Table Vl^l. Here^ PV classrooDGs 
were matched with com^^^txson clasmtooms on. ^ieven aagregate 
background characteristics. The forn?at"vi^r this half of 
th^ table is the same. as for the- first half* "f^re^^re^ 
however^ a few differences in results^ Firstj, note that 
the usatchmg was much less effective here especially for the 
three pre-- test variables that were included iin the match- 
ing variables for sa.mple 1* Thus^ the two correiatio:is 
for matched PSl classrqops for this santple are 0*2$ and 
0,48 while for the other sairsple they were Q.B2 and 0*860 
Second^ note that ty and large the pre-test mean differencoi; 
(column 3) are v%:ty siiruiar to the mean differences in 
sample 1* The chief exception to this is the PSl difference 
for the Level III siiqb Canaiysxs 2K The difference bet- 
ween the PV and Cor.f/ar ^son ,^^atehed .ineaas i^^ roughly 0^92 
points for thu sample and only points "for sample I. 



Looteig now 'at the estiisait^d aiffer^ncesit one© agrain 
th4m lyre stafeistically significant differences favoring* 
^ thf ;COj^arison group for Book 30 -an -th© I»evel III sites ■ • 
Jafai^sis 21, - All tiiree of the ©stiisated dif ferences / - ■ ., .-'^ 
■ are* statiiti^ai'ty -significant 'at t^^^^ 0.05 -level, A ; . 
sfee©i4d gift of consistently, signifieant restilts occar 

■ - ■ ■ ^ ' - ■■ ■ ■■■■■■ 

for. t;,he ^Staf|f»rd«Binet, this time; 'favoring the P¥- group * _ 

\tu contrast to tlie%ook 30- effects; hot^ever, these,; ' 

: • ■ . estiroat€is seem -to toe ii^fluenced a ■great deal. "by the- 

. the reliability; ''corrections'* carried but on the cdvariate* 

'■■ ' ... Given the high estimated reliabiiifet of ' the Stanfori- 

. ■ ■' ■ . ., ■ ' - 

Sljnet on classroom means (roughly 0.94)' and^the low 

' ifttercorrelafcibn isetween the two gro^ . -• 

. ■ \; - for the fire»test: the« internal, reliability' of .the- covariate 

•is probablf' close to 0,85 . (see^ f ©rjaula in .preceding _ 

- ^ feotnsstel,' This indicates that: the best estimate, of the 

• dif ferences' is g>robably around :.2,0 points or roughly 

one-*!slxfcfe Qi _the standard -deviation of Stanford-linefc , 

, V individoal post-test, scores s This est ipate' is .roughly 

double the #stiriated,:dif ferine© found f©r the Stanford- • 

Binefe ill tfte ' first- saaipl©. anaXf si©* . . ' ^ ■ -• 

- ' • f'here ■are:t^A'©. ©ther ©tatistieallf ^^^^^^ 

. ' tmmmm m tMm Tafele* Vot bbtu analyses ©n 'sanif^le 2, tte 

' ' V§l Milm&'iifsef, mm m -th^ rellafeilitf . ©stipjate' f ©r ' 

the c@varia;ieff m statisfcEejilIf sifnilieaRt lav©riaf thm 

. '. mwpmi^kseu fr©up« These differences' C©f reiifliiy lo© and. 

--"Moi psinfesj' ar© feefcMeeis '©'=1© -snd isfcaniafd digufiati0ii-2. 

ERIC ol-th© PSI post-test for. ihdividaa-lSo The high' r®liabili-fcf 
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of the PSI aggregate classroom scores and the relatively 
low correlation between laatched PV and Comparison pre-test 
classroom means probably indicates that the "best* 
estimate of the reliability of the covariate is 0.80 — 
a value which does not produce statistically significant 
iiifferences between the PV and Comparison means. 

Conclusions: 
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Our conclusions from this chapter are quite simple. 
By arid large/ we find no important differences between^.tbe^ '^ 
PV andp Comparison groups in their overall effects on the 
measured outcomes. We reach this conclusion in spite of 
the fact that in one analysis or another there are 
statistically significant differences between the two 
groups for.each of the outcome measures . Consider the 
measures one at a time, for analyses of overall differences 
among the py and Comparison groups. 

iiy gsis Though there were statistically significant 
•resuit/s favoring the Comparison group in a few of .the 'cofi" ' 
trasfs made in this chapter, the vast is.ajorifcy of the con- ' 
tragts were not stata-stically significant." Nonetheless, 
■many ®f .tte estiisjited differences between .the groups were 
in the general area of §.75 pQintm favoring, the, mmp&rimn - 
qmupn If we were to make a best het on some real dif« \ ■ 
■ference featween th& PV and c©ppaif*son saiaple w© would guess ■ ■ 
that the cmp&timm GhiMmtig on the average, omp&si&mmd - 
the PV cfiiMmn fe-^ roughly 0»?S points, give or take a point. 
At the low end of this interval- the different© m negligible?*' 



at the high end it is roughly 20% of the standard deviation 
of S^Sl ^ost-test. . 

(•2) Book 3D i This is the only test where a modest 
case can be made for a consistent difference favoring the 
Comparison groups. Although no differences occurred 
in the contrast of raw "gains** or in the regression 
analyses, statistically significant differences occurred 

Trr^thr€fe"'of^''tHe"sIx sets -of"- 

analyses. All of the significant ^3ifferences favored the 
comparison group with the modal difference being roughly y 
0.4a points or 14% of the post-test standard deviation / 
for Book 3D. The largest estimated difference for this 
test wav^' roughly 0.50 points favoring the comparison 
group and the smallest roughly a zero difference. 

■"- (3) Book Ah t Estimated differences between the PV 
and Comparison.- groups for this test range from 0,027 points 
favoring the- comparison .group to-^ 0.44 points— a statistically . 
.■significant : difference favoriiif . the PV- group. In almost, 
--all in^stances the^dif ferenses were very small and insignifi- . 
eant thoufh they .fenerally, fmar the p»/ group* 

t4l ^to£.J£MMHSa'. Is no /instance in an analysis ■ 
-of overall "dif fereiiefss b&twmm the and- Comparison: 
gr©«p was iJ.eor.trast statisticaily signifieant for this 
mta&ure* Bf an^i' li>ffe thfi rajority ©£ differences favor... 
fche- fro-^f# with tm average difference fc-eing roufhly ' 

pmrAt. or S»lit of tft© pssfc-test staR«lar<i deviatiori. 
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<5) Stanford-Bine tt This is the only test for which 
an argument can be made that the PV group outperformed 

the Comparison group. The maximum difference between the 
two groups occurred in the analysis of differences between ob- 
servea -gains"-a difference of three points which is^^p^istically 
significant beyond the 0.001 level. The bulk of this 
difference is accounted for by one sub-group— blacks with 
no prior preschool experience who will enter kindergarten 
-gained* ten points more in PV than in Comparison classes. 
Ho statistically significant differences occurred in the 
regression analyses, the multivariate analyses of variance 
or in the analyses of sample 1 of the matched data. For 
the analyses in sample 2 of the matched data statistically 

fiignificant differences of roughly two points occurred 
favoring the PV group, with "the exception of the 

multivariate analyses of variance where there was a dif- 
ference of 0.422 points favDring the Comparison group all 
of the other ^differences favored the PV group with the 
average dlfterence being roughly 1*0 points.' ' ' 



/ 



/ 
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. Chapter VII 

SHORT-TERM EFFECTS OiP ELEVEN HEAD START 
PROGRAM MODELS 

Introduction ; , • 
This chapter contrasts the impacts of eieven Head 
Start curriculum models on five measured child outcome 
variables. Two broad analysis strategies are ^sed: 

1 

model effects are directly contrasted; and differential 
model effects are inferred by contrasting each model 
with a matching Comparison sample. The results of the 
analyses are presented in sections four through nine of 
this chapter: sections four through eight consider the 
five outcome measures separately and section nine summariees 
the results by model. Before we present the results, 
however, we will consider some expectations we bring to this 
study and their implications for our analysis and inter- 
pretation of the data. Section one presents our broad 
expectations or hypotheses about the data. Section two 
discusses the issues of Type I and Type II error— of 
finding differences when, in fact, there are no differences, 
and of finding no differences when, in fact, there arc 
differences. Section three outlines the procedures 
used for imalysis and interpretation in the chapter. 

During the past decade a massive amount of survey ovidonco 
has accumulated suf (jesting that existing variations in 



■ ■ ■ 'ICS-' . / 

' ■: /"■'■'. . ■ ■ . ' 

/ 

. ■ / ■ ■ " . ■ 

elementary and secondary school resources (including curriculum) 
bear little relationship to variations among children in their 
scores on standardized achievement and IQ tests. (See, 
for example, Jencks et al. , 1972; Hosteller and Moynihan/ 
1972? Coieroan et al. ,a966? ISR study, in press; Children 
and Their Primary Schools , 1967; Racial Isolation in the 
Public Schools / 1967; Averch et al,, 1972.) These works 
corroborate fifty years of experimental research which 
indicate that there are few differences among curricula 
in effectiveness. Reports from Follow Through Planned 
Variations e::sQ support the thesis that experimental 
manipulation of elementary school curricula produces " 
roughly uniform effects on children's standardized 
achievement test scores (see SRI, 1972) . , - 

Work with preschool curricula has not been as 
extensive though the trend is the same. Weikart, 1970, 
for example, found that throe different preschool curricula 
produced roughly equivalent short-term effects on 
. children »s test spores.* DeI*oren«o, 1969, in a study • 
of differi?nt prt'isehool curricula, found few important 
short"toi;ii(t'differ<jfictf£» in effects. Other investi'fators 
(Karnes, . 1968,, for exaisple) have found some evidence of - 

differential impacts but h©r samalea were small and her 

■ .■ / ■ ' 

*Whilo Wt>i.k»iirfc fo.iind equlvaltmt "fjains", it should be 
noted that they wi,?r« vor^, largo, j:mF*portinf our argu- 
ment, of tiie ©vorall ahoi't«tem effect of prc«ichooX^ 
experience. ■ ' • • 



results may have reflected sample biases. Finally, the first 
year results ^from Head Start Planned Variatkons (SRI/ 1971) 
seem to indicate that there are differential effects 
of types of preschool programs on children's test scores 
though the differences found were small and the investigators 
indicate that they may reflect uncontrolled biases in 
the samples. 

*;he basis of this past research, then, we did 
not initially expect to find many instances of differential 
curriculum effects oriv standardized tests. Two interpreta- 
tions of previous findings are relevant to this study and 
supported our expectations of few differences, the first 

* 

involves the degree to which different curricula actually 
alter the experiences of school children* The second 
stems from the limitations of standardized tests. 

With regard to the first wc stress that the finding 
that school or preschool variation in curriculum bears 
little relation to variations in- outcometi .-^oes not, .mean 
that schools have no effect. 

We find the data, presented in Chapter ' IV about the ' 
.effects Qt -pr^sGlmol versus im preschool; to be convinclnf-— 
as we find the art|u.iaent that without, scho.ol lew children ' ■ 
would learn long cliviraon. Howcjver, we also find compel I, s,ii<|-. 
the. arguBitmt that m a. gri»at oxtent prescthools, no matter 
what thfn.r- eurrtcu,la. are strxkinf,ly hoTOijemm . 
~^&i^-M»iiHL»SMl^iSIL^^ ^oz almost 

all Head Start trht.ldron the turhoei yeas, is roughly ISO ■ 



days 4^>ngr the "school" day is roughly four hours long, 
the adult Child ratio is roughly eight to one, the environ- 
ment is safe and pleasant and rich with opportunity. 
Moreover, roost preschool teachers are warm, love children, 
and have a sense that they are important to the general 
well-being of all children* These facts suggest that 
the gross similarities among preschools may. greatly over- 
shadow their differences. 

the relative homogeneity of preschool, environments 
can help to explain the insensitivity of standardized 
tests to existing variations in curricula. Generally, 
standardiased tests are required to have adequate psycho- 
metric properties for an entire tested sample. Thus, they 
must be appropriate to a wide rang^^ of individual differences 
among children and consequently be somewhat insensitive to 
subtle variations in experience* Moreover, standardized IQ 
tests like the Stanford-Binet are designed to measure stable 
traits which by definition only change under relatively 
•e-itreme. differences in conditions. 

Overall, then, w® should not be surprised that 
different preschools have similar effects on children's test s 
rfe suspect that, by and large, most standardized tests,, are 
affected by the gross' experienees 0f children, and are 
little affected by relatively miner variati«3ns in^: ^ ' ' " 
style* and strategy ©f teaching, /This relative hoi!i©» 
g«*neity does' mt mQm that pregehool® ar©^ mt impartant 



or that they do not have an effect on children. Rather, 
as assessed by standardized achievement aind IQ tests » it 
suggests that they will have roughly equal impacts on children. 

The authors of this report have been involved in 
evaluating the effects of schooling for the past five years. 
It would be misleading to assume that this experience has 
not influenced our expectations. In short, when we began 
the study we were very skeptical about whether different 
preschool curricula have different short-term or long-term 
impacts on standardized test scores. In part this skepticism 
has stayed with us. This led us to an overall initial 
i expectation: 

(Ij We expected to find no differences among the 
curricula in their impact on standardized test scores. 
Put another way, we anticipated that the data would permit 
rejection of the null hypothesis of no differences among models. 

Yet set against this skepticism are the experiences 
of trips by some of us to different sites , of discussions 
with spons0rs, and the firm belief that variations in 

enivronment have- an- /impact , on chiMren mA adults. 

Our trips md our discussions iiave convinced, many ef us 
that there are relatively major, differences in inputs 
amsnf^ the madels r- .inputs as gross as materialss and' . - 
as subtle as different ways tliafc-_ adults relate, to childrim, 
Much ©f the data for thxs conclusions is presented in the 

. reperi im j»gle^£gitatien-_of ; Flaimed Variatieiis., qfTg^?! . 

tfihile that report point® out that models are not 



as systematically different as we might have expected / 
from taiJcihg with spoijsprs, it also gives evidence that 
the choice of model substantially influences the every 
day activities of Children in. Head Start centers. 

..If we acknowledge that there are systematic differences 
among models in their inputs and that differences among 
environments do affect child outcomes, we ar& but a short 
way from rejecting our initial expectation. The next step " 
would be to argue that the environmental differences 
' fostered^y the variation in inputs among models bear' 
a relation to standardized test scores. 

Here our task is somewhat more difficult. For we 
have little idea how much variation in inputs is necessary 
to create differences in measured outputs. The only data 
we can dsav on has been summaristed in White , et al. , 19?-3- - ■ 
. These data tentatively suggest that. highly structured , - . 
■ school ■ programs «sing. reinforceiaent principles might - ' . 
have some not iceafel©. inpacft on test scores over and above 
.th© gress inpact of school, experieaees-. . Moreover, .White ■ ■ 
■ et al.. found, that .no other' differences in .. curricula make 
. ■'■ a netieealjle impact. 

As we indicated A,ft Ctiapttir .II, tisree fssadeis fit White 
' ■ /■ ;. deserijpta^a of a pefcentwliy "eff^cfcii?©** prograisi "coniversity ' 
©£ Or^gor., Unimr&itf ' mi Kansas' Pi ttsfeurghl , \. Other 
iftQdelg, ,itffi.i.le varyirsf "ifj : the^r- eisphasis ©n" aeadeitde teaehifig, 
d© not com: elose t© felaciuf the iiaportanee on structure 
■ .and aeadeiir.i,.c drill- that tli#se models, do, 

ERjC , Ill I \ ||; 



■ If accept qW general conclusion that most , 
variations in curricula will have little differential 
effect on test scores and adopt as a possibility White* » 
arguinent that emphasis on academic structure and drill 
toight have an ef fect^ ve can formulate a second^ more . , 
tentative expectation I v. . 

(2) We expect to find no differences iaetween the 
eight less academically-oriented Head Start models- on/short- 
term academic measures of output. Those Ikhree i5t del®, 
however, which stress academic drill and reinforcement 
principles/might appear ioor4 "effective" than t1|e others • 
on standardized measures of achievement!. ^ , >■ 

Of perhaps more importance* however > is jsoiaethilig 
imjplied by both expectations • We think it extremely 
unlike!/ that any of the models will be less effective - 
.than the comparison Head start programs ; By and l^ge 
conventional Head Start programs do not have a structured 
academic emphasis* fheir goals are broad and as Boyd 
C19€6) noted, Head Start directors "reveal a prefernence " 
for a supportive, unstructured, socialization program' 
rather than, a strii'etured, informational program.-'. 
In; this sense they are simlar to . the , eight >V modeli;, vihicli! 
..do, .not plact hmm^r 0pphas,$s ■ odi , aaadamic driil* Followir^g 
■ earlier arfumajttt th^r#£or#i w would . expact to find no 

■©fleets 'Of tha ^iflit ittodMs whi^h- do not stress ,apa4eiii€i- 



II. Type I and "Type 11 Egror ' • 

Given our geissral ejtpectation that wiXl b& few 

differences aroong models in effects on cognitive oiitcoroes, 
' w© are ineiined to toe skeptir^al about rejectincf a null 
hjrpothesis o£ no differences. This is particularly true 
If the data;sugg^sfc that soise'siodels are "less effective*' 
, than. th«(|.r comparison classes. Our skepticism suggests' 
a conservative strategy ♦ 'It suggests that we should 

minimijj© the chance of Type I error, of finding differences 

^ , ■ . / ' 

_ when no differences exist,- This strategy however* has 

, the;. disadvantage,- -given the fixed sizes • of the, safpl'es, of ! 
laaximizing the possilaility of l^pe..ll error — or not ■de- 
tecting differenceii- when, in fact, they do occur, ' 
. While a coiiservative strategy is suggested by oi^r ■ 
.•expectations, it might no^ be" fully Justified, 4: con*. / 
-.vincing argument say be- mde th&t previous researeli int© 

■ the effects, of . eurrieulufli variatim' stouM' ii@t detemiise ■ ■ 
^..the. str&tefy for analysis and. interpretation in this stu%„ • 
■■■lteafi;be-, argued, for #a£aip,ie, that m '^tMv mmdf ®i - ■ 

preschO0.l ciirricula has ever had /ass great a div#rs,lty of 

■ -"treatsTi^r«ts*' a& Planned .¥ari6tior*« ' Jkrtd therefore, mmi ' ■ 
though differences in "©f f@etiv-#ra«iss" did. not oeear m\ 
ether -..studie©, there, is «6 /'eeEjpellifif reason ts b&lim^ 

■that tlwf wmM^ tmt appe-ar im tliig- stm4f * \ -Mfreever* per«- 
„ haps researeiiers .tsaVe an ©feligat.iQfi'.tO- atteiapt -t® . ■ ' 
tfease difefgrmeesi ©at ©f data ratte.i:, thaii m,mmv'.,%fmm mmt 



We ii^cv ^yi^^painef.;^^ to t^e^^o ar^iy^^r^ti^ , and fey ^r^i 
they dc^ic-r"^.:n€^s tne ^rpro.^ch taief^ m thi^ ct^t^^^ma^n 

i^p«>T^ of Cr;i. i-ii^i^n , " If. t't<i':-, report* hc-wtiw-r ^ iva iactor j} 

mj(^%t fint-A:- -ci un?;f;.ti;.wn ^n- -:iS'u:-v v:". th*^ dai;^^ . .Tilt? ^i?co^iid 
fed^Si^a of r:c:rt- tort*:' ev^lv^ation- 

l^iiat^^i^ c:*:^f^ :^.t.tA tc --ath^^r Tvpa I or tvp^' SI ^-rr€=^r, felt 
th^-Mt let;!-^ ri^jfi;;^ ^fi^ i * ^Ki' f .5 Or -zJ i> \;bs t an t ^. 



mn^&:- ©Ifecuj wit'h@«.fe sliowissg slt©ifs-t?em ©aeis. The sam© 
tmlM ' -rndQlm that a^te^pfc to- ifif Xiienc©- the wa^ tliat . 
^cfeildren st^mmm and attain, iisfomafcioa* ,.lffeile ds 

mmpe&t- tmm mS^lm -to be l#&s>e£fect.ive« than issst , 
'btter ii©3e.ls shas conventional llea<J ,^ Start classes # „ ■ 
in tte 6li0st Ktts *?e feel tha-fc -siicli, a fi.^diiif GomM lead, _ to • 
pre^atmre sanelssioKS about" tfe,eir ev#Rtiial ^effect ivenesig^^^* 

M©,mf, tiieref®Jf©«/6e. aetir»f irfesponsibly'&y ia^^ 
eladifif thmm mMmM- ifi, ©er analfseso . Me caw Justify oar 
aetidus ©slf % -tfe, arfUiient . tisafe w© slioiild have iMematioi?^ 

ca'speiliaf ©nanfii. t©. permit @i4t .g©p.tm6tiiif .aosl©l-sV it, ■ 
is mt ees;peii.ir4f eneiiffe psmit; a liberal apprea^fe t© . 

ffp© I mms* ' - .- 

fwd linal g©itsts sli®iiM:be wMm,,- Mth hm&^m d0 \ 
with mm-iM&mm^md Miimmvms Am&m§ ettr*i©3ltia Wimtf y^f^ 

-a Rttll^tifpotliesis €©es wMim mm:, hmm Ammmmm^ ■ 
tSidt- til© re d.re a© cli-f fes^^ses C@k. tte ^itedsiir^&t femitsi smsm$ 



*Ofa© ass-assptism. ©f ^prorJiastisri! fisBeti^W reft #f i??** ty: 

teifif mriflei ©at hmre 4s tfei^t all ®f tfe diffejr-smt m^^lm 

■Cleaelf^/ tii© faet tiiat si©se Ir*^d©I^s f®f. siict®sttgtiiR«! s# 
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fcte rs©d©is« it sinpi'/ st-^Jis that we d,© 
rar*fe ©f ©atsaise Ksasares Is gevcr^l*/ iiitittal ©ili'f@rei5fe 



tills %#a.y we fiape tie -afelt? m 4&mtm%m ■ f stinates 

ar«3 s*©Jisitii?£' m mM&tmit malftis ^ippmmh&G whmh 

ws, 1 1 spefisi. itpsrs' tism mf Mm fc^ inwrpr^fc i^iy tmu^k ■ 

Tte s'tratftfjiea m.Si fee- .usei ii^lfcafsesafiif o Me 
%<T3) fee; able- aa5^'rpr©teifc.i!gfis a£ else fcli©???*^ 




mt. a iici^ wBtm-^i;- ^©51* Smcic^ am not iicr^y^^^ ■ 



.^this m»f- reflect <am.^mum^ .diitemm&^ in t^e ^mplm 
ln^i<S'%md fef the Ui^t pw&t^cmmQf chiMmn In fcii© 
gssdel fr®4ir^» This .mt&rpmmti&n is ©fc$efiftn©Kecl >#te^ 

see thafe fs-^del^A sts sifw^Cicarifely Fisre ef festive, iisars 

, ws3akl<3 fee tinafe .ir^del A is prefe^tblf ^ot «*iT?©r€'' effect iv©* 



ERIC 



isiiml^^^mm effe^&ive* i?i .fcf^tft analyses, f^mr. 

the ©fiaets of tte ©Jfifirial afialfs.iS. wefaM t*®, weakened 
■una ©ar c©ii©lsisi:@i4 aiglit te& fe^at f^fiel A m iMmid "sere*/ 

mm-mMi^mrmd With tha. mqnit-M€i mfi^^m. a^tm^tmi nfm 
tkmsf' mmi^^msf ' turn mtli t«fifj«" isdieafef pwmim ■ , 
pr©tei^bAlity f©r .diffei^ReecS;, mfeh data %fliiea pmfe^sfely 

«fe list fch© assiaiRpe.4©fi-s tm mm^iMsmm. tfestifig. ■■■ 



gitea »n %im tin^l analysis sar#l«J. The means mm . . 
wnweiftitcfi -dver^gci'S ©f the -elarisrcfom F«ajns» ■Tables 2, 
aricJ 4 f€*f eacsh ©-sitii^jf!® mfs.mm& €or»tair« ©ffecfc estimfees*. 
Fs'ar -sets of arsaljfaes. ar© ?cipir€?s^rit<s«3« fli© firs* as«s 
slie m tim mat ©f ai5i-al.fsii5«»^tw© analyses arc? ,• 

',j!!io#^i wi'fcsv .ifc6Nf&i?..pafris©ri ffs^iipo contrasts tlie 

mm&Q in Chapter it,-, tim ''etfmt mUmmt*/" 

-4[<PV psst ff&afi » P¥ f»ris-tfsfc fifeafti - Wmmfison psst-fcest 
isea^-i^ C©i6gai?is©fi pre-tfst mmPlW this is. a siiKpl© - 

is ^al€i^lat€'d iit a C9ispi« parm-sr.bat the is 

sirdiar*^, Tr© «estii?ar,e ©gii-als? - CI IF/ f»st..-test. ©bsert^«?«l 

&¥ip€'S%eS- ^hmtmx .j^a^' .©f , eicpr©sgi»f tfiis 4S^ ta . 

be.thout*** '^^^ t|yi>.cvJaissi.'ailaf \to^tS|#|t»^^^^^^ since 



».■.-■ ■ ■ ■ * ■ 
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fchcW© is on^ Conparlsoii site for X\m two sites 

dRiitfsies p;ay, %M&mfm%, m>fc tee valid . for th© Far Wijst 
- "Kiodel* ..piir^e dafca are prc^sigiifcciil in fehct mmn^ tafoXii far- 

Tft# si^eonil ' s€^t of ■^effect <i^.tlmafcc^s'''' steins fifciiyi a 
ciir^rfc m^pariison of ' PV elasar^^ope within P^^d^la in 
^^^^^W^mtlate analysis ot ^^^ai^iance Ira^eworli^. CQr^mismn 

^ImBmmt^wBtr -ihiQ {ippxQmhm mm umd^ Om? fcrc^ati? all cif, 

Cc^isp^arit^oii. classas tofc^tfer '^\s "a r/jdel*" "-and fcc;*stf^ 
for ^Mtt^mmm h^t^tm^n tte W «i3els ^r^cl tte overall. ■ 

immmmkr, The? ^tter tates P¥ uricI tT^mpsiririon ^roaps hf 
it^dels iisiiif'Oiily thosii' sites.' mmtain b&th FV m& ' 

: copparis©!!^ Qlmmmm^^ ■ A i^iisit i¥ar iirte anai^r^^is of c^^rarianc^ 
icj iisigdi and om^ 4^qr&0' f rii€'cl«rr e©iitrac^t# fc#>l?^<^f£-ri ami' 

■ C^ii^p^risoi^? -c^las^^^osi' ar^^ ^^rti^iA ciyt for' c^ach" Data 
^ froi^ the i=j*ec^eiid''aiiid thtrd setri of ^nal^f^es M'm in %lm 

^ ' Tte f^pytrtli iabl# d-isplays data £r» the aiwly^^es M ^ • 
^pafec^lied'* P¥ and Campari s^i^ ^las^^^&^s^' ife presc^fit c^foserwd . 
and cc^variar^ee adjui^fceci dif fc^rei^^es fa^ poitel fc^r to@tti tte 

Our approach -to intarpri^^-atici^ii c^f iiie data i^;- 




First, we GQmidQT sit« io Bi%& diMemiiCQB in ,pr«?« , 
test fweans and ^aitm im both PV awl Cfmimwmon qmups, 
Tim f©eu» s,s on th© disfcribution of fete ' data— extrefR€» ■ 
eases and the rang© of, s«©r«6 'between the .2Sth and 75tli 
p&tmntilm:* ar«s ififJieateti*. Seeendf w@ €5cmsid«jir imKk>l to : 
irK3d«l di f f *?ren«#3s iji FV.aiid C©pparis©.n gmups for/tocith, 
"observed g.s4ns" For *'qb8i?rwd»exp«efc<?d tains'^ 

/Third wt? eotisifter' tiM?/*'adj«f5ted'' dift«r«iees 'between 

point Gut pattrnm €ii results and fc© ©Epl^in iiiecmsistc^nei^a 
in the data, , Pj-n-ally piirifRari;?.*? th»? data for the test 
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IV. agok_3P ,, ■ 

Table Vll-l displays Book 3D pre-fcest scores and gainii 
by Bite fo^ ^oth PV ami Comparison gr&ups. The Kaw|<» of 
I»V i»rii-.t:«sfc nreaiifi is 'tmm 8. a t€» Erwfbly two indiviikal 

l^vel standard 4©viatx««i {sc?<? Chapter X 11) . The? eang© ^ 
0f Coftiparirjoii pro-tost, means is soiwjwhafe swall<?r--f.rom . 

^,xa,ji..i^jyi,.a*-.-..JEh^^^ 

have a ranf© ©f than few© |j&irits«*froin 10, t4 tc* 

,ia,7S. , The iniddle fifty peccfi^nfc «f fe^^^^^^ 
■sites al6© dlCfee bf. less than |iolnfcs rangiwi tg&m 
means ©£ ll»OC» te l2«7So , ^ . 

■ m site rang:c! fifop »0*14 /fc=i3 a«8fii» WMighly 

1,4 l-ndlvid-aal sfcandar«l cl©vi*itions. The range?. f« ' 
C«ifiparir*cm sit© . ''fains'* .is swallef , . foinf freiM 0»M tci 
■3.44 pe»iiils# P-r mmt mm Awiiviaiwl stamlard tlwiafeicm* 
When w«? loefe at tlw PifMle f iffcy .pejeeept *rf the, PV. 

■pgitifefi* fhe CowparMion miadle fifty peifeent range is 

• s/nfJ« «f- a«||k^*>«:|at«? si fees wi tAiii isQctels and 
intfMueG ¥ari.alAes m mmmmtm, tefch fc€Chftic|tt©s whieh 
qt*j»€Jrally lead to a mAmt:mn in m.ttmtmm&&, ttie tiite. fc© - 
' tmm' iikc'i? ^ larg^ ap wi^ wiU ^t^c^ for 

dif fcMfci'iic^c^ 1.11 ""'fains*' i^r^-cafii c^^pc^^ft 'to fif^d oponf - g*^^^P® 
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in our' ano^lysiiS roi^glily 1*4 individual standard devi-* 
atJLoiis* When P¥ sites are Icolced at within leodelLs the 
mtteet Qi affregating heeomes a^pairent* the'^wt»im% with 
the site *§aitilng* the ieast is EDC« ■ fet both of the . 
otlier E0C sitesi y-ield *9ains" in the middle fifty percent ' 
range. And tht n^odel ^luntaininf the site with the. largest 
•qain* CKigh/S^^pel also ineludos a site whose •gAiii" 

iBorc than one 14 te 'in the top twenty-five percent and 
only one leodei'^ IBmk Stm.&t) has tw© .sites in the laottois 
■^twenty-five |!e|^2enfc, .the ' variation in "gains* within 
woclels, theref*'r«i, mskUm rondel to.cwdel differences con-- 
iiderably sm per than site to site differences, 

, Colmni I .and 2, in table VI 1-2 clearly malees this point • 
iiodel ipf Jtn "faias* s«5. t»is taisle vary froa l,4f points 
to 2.95 p#ints*-a spread m£ roaghly 0»5 individual standard 
deviatiefa*- c©i?|sari.s@s« "isisffcl^ mmn "gams* also have a, 

range .©* reufhly @t.5 4ii«ii¥.i.<3-Mal ©tawlari deviations # frop 
- I.i2 tf 3,§^ poi!i%s„ 

. I -ten di£lert?iie©s feetwef^n P%- and Cemparision f roups 
%fi^th*» MS'iiel ^isee eonsi^iC'rei tM» pm^itf ssf larfe ^ifferenees 

tqtfiet^ modtil P.eaiis* Ttou® * • th^ir^ may some si^all. 
iljoniiist^iiit remit J if tSe ri^aii^-r compares thm tabids*".. 




io Ml miMtm in ms^c- sin ^mpt^ mm mM 



becbisiQs evea ele&tmt^* Colwm 3 shows diffieirences between 
PV and CQsipaijjtsoR giroup *gaias* within isedels.* f h© 
diffeceaces 4« "grains* range from »3.62 goiats. ^weirinf 
a Conpariseu qm^p) to points CfavojfiRf a W iiod^lj. 
Jioke o'f the dif fesefkaes are statistically sifiiificaixt. 
Moireover, tsie- largest dif f er«r»e-^,^fa¥oi'in«f ■ the coiaparison ■ 
1 ffo«p wMf bm ' dnm to an ;iRap|>r©priate ^.ateli between mnd. 
Cog^arisQ^ fEou^sins© it "arises, in -a siedel where there 
were m sites aad.oRi^ ontf e<9!Bpar£s«sii site Ca?i *'off«sit^'' 
CQFspactsoR at t^.atK ' ^: ' 

a^d Eitpesfced"' msm fains for tlie FV ani eo^parisoii froijtps 
wifcliin mM&lmm The rans© tSif ferenees is Iron 
favor iKf th© Cojiparison group t@ 0of2 , faverin^ th# P¥ 
9r©ag,, In this insfcancet th© intrMwateion ©f s©r.tr@l ' 
variafeles 3iiier©ai!©i the sprediS ©f ditfmmmm^ to© of tfee - 
dif foresees is statistically sifswitieant -ffer the ht.rmuA 

this sifRilicafit fii^diiiif 'is flifli©islt# however* sine® it 
may fee t© iftapBropriatc- C©r»pa#is©fi fr@isps» F<©r ©Rly 
©lie ttie t,'W© site-s isi ' the" ArizoRa rttt-iel fchers 
a Coisparisim ss,te* and it is a» ©ff«sii'© Copparisoa, l!or©« 
©ir*er* 0»e @f Mttmi&^B P¥ sites lias ''f-iiris'^ isi ttie^ t©p 
twe«tf*fA¥e percefst. ©f.the sit© gain dig fcrifeatisa while /' - 
the mtmfs had , "^f aias* in tte siidl© fiSty ' pc-feefit» tlm ■ 

^of £«site" eempariiaeri fr@ap had $hc? -fsisrfeh larfssi; ^qAtf'' 

... ■ ■ / ■ • 



of the Cor*pdrison grottps ending up with aa "^JbseyvM** post- 
test m^n of • 16,75, roughly three points above its y'expeete^* 
pQSt'-tept score, this was the largest difference^' hi^tWeen" - - 
'observed* and "ea^ected" post-test site leans.'. It 
seeiiSi; then, that in. this analysis , the Arizona; JPV kites '- . 
had/the ?^sfortune of being contrnsted with one particiJl|irly , 
ef fective Cojaparison site. To conclude fro» this data' 

■ ' <^ "... , 

that the J^izona taodei 4s not as. ^effective* as 'othe.r 
iSQdels'©r as smst convm%imtSI''uSSS^£mt"pm 

fee .a isista&ii..: ^ ■■ . . •■ ', . " . 

JUlOther ^waf: of vloc^kinf at th©s« data/ sagger , 
... . «< . . . f 

there ©if lit- %e larger differences ansonf the i^els, than 

we have indit^ated* If, ..for exaisple, we contrasted tite " . 

dif fereneess between the ;tsost "effective*, l^V csodel . Cjadged 

by I contrasting ■ mth. its' Coisj^arison group) w£t£.. the leas fc 

"effective* W nodel, we obtain a. difference of 1^83 , ' ■ . 

pointk C0t.S2. * -«fl) , JBut .©ven thiS; dif fe.ren<ge ,is oaXf 

roughlf » standard deviatians-'-not _ trivial but certainly ■ 



*lfe ■ mil " use tte .. t'tntis'- '^'^^^l^ast^^. , Mfi^renmmm'^ ^emtimA^^ - ' 

-We. Will ■ii^€?*jiss the^' results of th^ Boole •-SP aixalys^s Ib- a #0*^^ 
■."^inprafeitiisiw f , than thmf ^mmve^^bf: '^Qinq m ^"^m-'fmpm 

t4 hm ablm^ t0-^ iaiiiliatizm. th^ ''-mMmt^-' Mith ^ mm^ ^^f' tte analyse: 
fer eaehi outcome jpeasur^^* . , " 



-181-,, . , ■ ■ • 

, .=■• ■■ ' - . . ; " , . " ' . . ' * • 

. analyses ef the^ Book m data.a' 'SM mmlm of ItS eDntrast-s 
m;^ presented. Five contrasts ' if i®M statisticallv . si^nif i- 
.■.caint results. ; Fomr .dff ffeit^nt tf pes of -analyses; are regee- « 
■ :\ ■ .sented Jirt tlie 108 contrasts. • ^ eopplet© s^Msaarf the ■ 
■ V data- «ouW prodijce a litany of /nsisall and insi,fnificanfe ; 
■'-■V .. , "differences;**. Instead of foinf titat rotate we will bwmilf 
interpret tli© four sets of -analf .ansS then ©aant oiit ■■ 
._ * few i^atterns in the %ta» : ■■■■ 



dM f^rences ' between FV ssedel -i5.iap.s- and an inweiffteed ,.. ^ . ■ ' ■ , y • 
V overall/ grand ;jsean».^ ; me teelimga© ^o^^^ , / i. . . ,. 

.siultif ariate analysis of" eswriap.ee « \ f l|© ,'4v©rall tins^vari- ' . 

ate r for msk 3& is 0..t'-t44r'' ifiii«ati?^f ,tliat ,tli# dif.ferencfs;^.- " • - 
■J"' ;\ '"-an^siif tte i?sd@is. ®otild easily %mft mmkmA fef'eHaiicf*^ i : 

■. -"If tfesre. art .ns 'triie* -differences., .tlofi© ©f- tte "effeet - " . 

- .. •■ ' ■|2f"'e(§difes'a and' 3 ste^M ^dSiastM dilf€sfei5s^^ • . 

:.. ' : m& pQS^M Q&m^^^^ 4 -sepa-rat©. . ' !;, . . 

msi-iU&mmm i&w msi. ^&wmmm'S:-mBt pr©-3€@r© :mfi - ' • . 

'''wi£h s£a&4sti.eaJly s^ifnif ^-^saKit emifi&imt^ afi4ji#.3ti!if ~ . , 

d® net watit t@ five tlie ijr^s^ss.icjsa ttet tse-lSi ssptr-as^fea a&e r 
■ ^ all ip^eseiMJeiit. Sl#arlt ' to ^©fc^ ter ptirpsi^^ife w^psrtsraf ■ 

PRir .the' narij^r ©f statist iealiy -SigfJif-icaiit- s®i^;cr*st3 1*^; siri^Ir ■ ., 
give the/reader, an overall £a-rK|| fapjjeisAor,::'©.! . thfe'data. . . ■ ' . 



m^^.l in M m^h%i^ wli^^e all .t^# leoisifarijien clatr^js mire peoloa 

j^a^i jr^vi^ft.^ ani f.li^ ^M^^mt^d*^ Cot^^vinm "^mdrnV^ mmnm, Col'^f^^ 2 mhmB 
tise. e^^f fi«s53^ts for ^ x^^zmmlQn m^limXM not ailowinf for nepatatii " 

ricii^c^nfes £ot tte covariates for the di^fferent iispdeis* Col^s^ 
.3 shells turn cmtiimQntn mllminq'- for »epar«fee m©4el waff Icie^tE* • for ■; 
fche pr^-£€?s-t. m4 im pereent prior p.i:a»ch0ol* Colism 4 ihws the-, 
dilfete-r^ b0mmn wv.-md Coi^Sitxmn qxonp "^^dimted*" mmm within ■ 
md^lB mt iiitf^B wia both a PV^md ^ C^^^tiMbn ^mtip* fhe estimte^g. 
am I ^K^m^- M fr«d^}^ aoiutrasts in the fras5«ework bf .a one waf .JC?C01?^,_- 

A gsot^^ io^ll-mim -te ^t^l^ list0 thi>. cm^ti^t^B in the asialy^«^.. . 
to , all ©r,4lyf.i?0 ' fe^e classr0Ois5 is tfe miCcs-f analf^iV^ See t^ag:^ ^ \^ - - ■ 
tern V md vtl) iov furfeM^r diaeassioB .of th^ apg taad ^-^ 
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*"'-^ Statistically signific^i at, -thk .0^ lev©l- 
** 0 Statistically sigjiifiyant at' the • 01 level 
*♦* Statistically slgnifCcant at. the .001 level 

1. v dnly classrooras/are inclufa^ iii this analysis. 
The multivariate # wiyCh the PSI, Book 3D, and Book 4A 
in the analysis is 2/36? significant at the .001 level. 
The overall laultiva/iate F for Book 3D is 0.87, which 
is not stat^fStically significant. 

2. Both analyses/wer^. in the regression framework with 
the pooled Comparison Classrooms as the "duamty variable" 
left out of the /regression. Analysis 1 did not contain 
separate slope coefficients for the various models. 
Analysis 2 allowed for Wparate slope coefficients for 
PSi pre-score 4nd prior \ preschool experience. Analysis 
1 escplained 7|L»3i of the total variation? analysis -2 
explained 76yO% of the tktal variation. 

3. Only sites with both W and Comparison classrooms 
(on or off-lite) were included in this analysis. 

Notes All/ analyses included the fallowing covariables: 
PSI pre-t^st mean. Book 3D\pre-tesii^ mean. Book 4A pre- 
test -wiean/ mean age> percent black,\ percent Mexican- 
American/ percent female, mean income, mean household 
size, teacher experience in \Head Start, teacher certifi- 
cation, mean mother »s educaeibn, percent prior preschool, 
average /staff working conditions, whether the site is El 
or Ek* /in the analyses in column 1 the variable "site 
adminisftered by CAP or by Public Schodl was also included. 
In' the /regression analyses in\colijmns .2 and 3 teacher 
race Wj^s included. In analyses of the\ Stanford-Binet, 
the S^anford'-Binet pre-tfest w4s ali^o ihcluded as a covari- 
ate--these analyses used only iLevei Ili sites. In 
analy/ises of the Motor Inhibition only classrooms )irith .- 
valid Motor Inhibition scores for both- fall and spring 
'were/ included; " ' 



that fojr some models tJi© r«lati©»sfaip between the eovariates 
atid B©o|c 3D differed txom thp ©v©irall pooled relationship. 
The e«4ct'of th© separate- sl©pe eoeffieieats was to change 
some o^ the **adji2st©d" dif£®rfeiic|es'' between th# model tens 
and thi pooled. Comparison i^ean, 



Specifically separate 



seere entered for Bante-T 



slope ii©@f£ieients for FSI p^r« 

I ■ \ ' - ft 

Street i University of oregon\aud for the Enablers iss^el-, 

in each of - these" cases the "ad, lusted difference" .for: the 

model changed rather substanti illy though in none 

three cases did it reach statistical significance 

two- other models, however/ the effect of the new cov^aj?- 

iates slightly changed their relationship to. the Con^ari 



son gr|>ttp, shifting them from 
in the, colimrt 3 regression to 



maanitiude in the Column 4 ana:^sis. The University |f 



Floridia and EDC models have adjusted means" signifl|^tly 
smaller than the "adjusted Comparison'* mean at th^|4 05 



a rion-s/ignificant statf^' 
a- statistically significant 



'-level-. - - . 

(3) Column 4 of Table WI-3 shows one degree free- 
dom contrasts between PV and ;oroparison model means || jus ted 
for covariates. The sample represents only those PVpnd . 
Comparison, clasps in sites with both" PV and Compari^ 
classes.* None of the contraLts are statistically sii|ificant. 



■ *T hus, the contrast between j^le ^ftx^ zpna ro^^ 



"c^^Faiison mean,' Uses 6nir«^epr ^^^f^^'-'' ,^^^^11^1- 
the Comparisoi) classes in Lafayette's off-sxte Comparison, 

Albany.' 



'.. (4) ~ Tables VIl^Ah asi<l 4B show data fjf©m matefeed WV 
and ■Comparison classroom analyses* In Table Vll-Ah nom 
of : the contrasts"* "b©tw©eE W mod©! m&mB and their 
matched Coaparisoa /elass means is stattistically signifi-" 
cant '(see eoluims '4-61 . In Table VII-4B ona set of con- 
trasts shows sif nif icant results. - In these analyses it 
appears' as if the High/Scop© model is significantly more 
effective than- its Coii^arison • classes . No other con- 
trasts are statistically significant. 

The very s mll percentafe o-f statistioally signifi- 
cant-contrasts iA thes^. tables, the lack yof .any robiast 
"effects*/ (icodel^ that show significan t resnlts in a 
variety of analys| si^^ and the overall siiaiiarity of the 
observed gains fp| the different m models and Con^arison 
groups > suggest 4|at there are few important differences 
among fch^-H aad Stjrfc curricula in their effects on the _ 
Book 3D telst. 

There are, however # a numbsr of patterns in the data 
which can be reported. ' 

(1) , First* we find so data to support our "tenta^rive 
expectation about - the specita. effectiveness of highly 
structured, academically oriented models! No contrast 
involving these models showed statistically significant re 
suits. Although in all three of the models (University of 
Kansas^ University of Oregon aM University of Pittsburgh) 
all PV sites fell in the middle fifty percent or upper 



'601^4 i Hsh^s tiia i^pifeer of' matched of «?im»|i:o^p tm 

the^^da^. Col«» a ihanK the m^ml^%tk .foi? ;«*cb »»a«l 

the una4j^£^tea d^pendaiit variabie i^aht f or «ieh »Od#l t»V 

€. »hO¥ ad'ja^terf icpendent vatiablof^ for «*eh nediil J^f 
adjf«tca im tHa covaifxat©^ nmd«r fchr<5^ condition! of « fci*- 
mates fefie r^liabiXity of tiaa covariac* {ooluam 3 fttimto* 
ific: rellaiility as loOO, eoTom 4 a» 0.96 and c^oiuwn S «• 
0*«0) « f 1:^ Itotd-I^orfear ^soirafiscfeiosi ia uaad to *cori:#ct* %m 
eovariate:for Jettiiabilit^f^. ' • ■ . 




- CoiJ5^» Covariati^^ \ Covariafee \ <?ovaria^e 




- -^-"-Stati^li^sraXIy signif i«?anirSt this lavai ^. - 

■ Sfcaiisti<2?aJ.ly significant at tha ^01 leval . - . 

♦**StatisticaUy fiifaJLli^aiftt at th0 *00l lavol 

^The ovcsrail eorrelafcioft fe^Aween I^V pre- and Coit^ari^on pre**te»t 
mateh^d elaissroom masarcil ♦||i* overall F for the teat of 
homcg«2naity of th^ covariat^ regroasion coefficient 3#07 , 
' jf^^j^oasioh coefficient for the covariate for the analy«ia with 

reliability ^sr^timtcd aa 1*00 « -^Ojii* ^^^^ r^t oetimted aa 

O.fiO tte eo0fiielant » -.14^ for t^x. * OTSOt *^he coefficient « 




■ Selected Sfei^eiibiiaa"'!©^' Mat<&h0i ClaQaife^?^ to^lf^ls 



tte c^l^m .1' nhm^ ^tia TOVa^lat^ mtanii.- for stci^ mmi 

i nhm¥ M$mt^4 d^pmdmt mi^kskhlm mt mcIj »cdai 
^muBti^d for tm .&wmtxm(&'l mmr thtm cronaiUons ««^^' 

the mliiihlUt^ m i.05f ©olttmm 4 as mm f&lmn S •» 



P0^t->fmt mio « loOO IUiI*^» OaSOI K^i^ t.,^^ 




mtehedi C5:iae0rom maBumn « 0^^* Thi^ oveirall F for tho teat oc 
hommnoity pt tha eoviirlafe<s risgiroawion cioofficionfc « ljii.<^ 
%hf» raOTcauion «?a^fricJL0nt for tha c^va^iata for tha aiialmla vith 



fieoups did - almost as o'H^' Cop^parlson site af 

lor thim qtoup.Qi mti#l@ tmll In %hm bottom twenty- 
ilvm pmtmut ii\ terms ©f raw *'fai^s^» Wten we loofe 
modal l^val data we find tliat# on tli# average ^ fete Kansas 
a'lid Pittsburgh P¥. classes gaimefi sllfhfcly »re than their 
Comp^rmQu groups, iii terns of, both ^. **obsesi/^d^ gains and 
^obsfe^rvad^e^cpactM'^^ gains, whil^ th^ Onivarsity of Oregon 
i»odal onl^ beM its own with its Comparisoiif » This 
pattern t however # doas not l^old for the di£f^r#nt analyses 
Oregon-i-for ea^anpla^ appeases above average ie the ai^alysis 
diraatly aompariiif ssDd<ils# equal to or slightly above 
average in the cfompariaons with the pooled Comparison 
classes r almost tiKaatXy ' average in tha one degree of f r^^-* ^ 
dam contrasts with Coi^parison elass^s in -tfea sanoia sites # 
slightly above average in.tha fivs faotor mtehirig analysas 
and just aboyt ai.*erag4^ ^in the four factor matching analyses 
Tha? (istimat^is of ^ th^ ef jfectivanass of the Kansas model \mrY 
somewhat In tha 4.i^'^^^t. aontrasts among w^dmlm 

Kansas appears avig^rage/'- in the analyses with the poolM 
Comparisan classes it appears hmlmt averaga# in the eon'- 
trast' with Comparison groujps in the umm sites it is 
slightly balow avoraga, .in the five factor* match it is 
rottgil|ly average and in the four factor matched raaalyses 
j- fc-" 15 sl ' 4 #rt:.T y fifrovf^ .^^/ j^ra g^f " Pttt fi burgh f i ppf ^ ar^ ^ligh fely- - 
above average in tha direct contrasts among PV classroDsns. 



mud in thm ea^trmsts against thm paolad Comparison olass-* 
TOOPS 9 ■ about '^w^raf a in thm onm dmgxmm of freedom aontrasft ■ 
with it© C©ii5p%isoa sifc## and slightly below average in . 
the . analyses of ^ the tMQ^ imatiehad sannpla aaalysits* Certainly 
this^i^ data offsi\ rio eupDort for our texitativa expectation « 

12) Pour 0iher models > (Far Weat Laboratory # htimmi^^ 
Bank Str^et^ and tte Bnablesul "show liiaced- patterna ©f re-- 
sliitSo ' With the riic^^ption of the Arizona ii^del th^rei are 
no statistically sifBif icamfc 'Oontrasts for any ot thfse 
ift^^els.*— th# on^ sifiiif leant ritstilt for J^ri^o^^was dis«- • 
cussed earlier* On th#- feaisis of ttei?© data we mm no 
reason to argue tb^t any of these .isodiilf ^dliffer from, the 
average in effeetiVantess on tha-Bofels: Id* test, ■ ' ' 

(3) fwo other ittodels CHigh/Steop^ and'KEC) also show 
miKM patterns of results hA xn mmh instanoe -soma set of 
contrasts is of saf f ioient i^gnitud^ to .Reserve attention # 
^hm High/Scope i?iD«Sal is partiaularly interesting^ . Thrse 
PV 'sitas ar^ iriolMad in tha/'ahalysas* Qn thm average , 
tha pr©**tast means for the Migh/Scopit -FV sites ara beloir 
the overall Coi?iparisoii maaiif th# mean for '^eir own on 
locat ion Comparison si tea # and ^the ovarall P^\inean# Two of 
the sites .havs the largest, abservad gains l%the sample 

whila the third site ranMs in the bottom tw^ilfcy-five par- 

■ ■ ■ . ' . ^. 

cent of obsarv^d gains, All thre.^ Cpmpari uon lii tes' for* 
this lEodal • hava . mvtrage **otoSi^V€d" gains* "'\ 

In the^^coiparTson^ of /'obsarv^d^ gains the ifigh/Scppa. 



■■W sites' GQWM out looking i©ffi©wliat bmttmw ttoii their 
Coisparisoa sit©p mmn thomgh th©r© is om ^w©ak^ m sit©. 
However, wlieis "aaSustf^a. p©st-t©gt" ffi©aiis f©r.,|.fefe sites 
^re ccntrasted t© ©ifchgr Cal thmtt. owa Cenparison site 
"adjusted post^test Cb| t® th© overall C^paris©is 

group "adjustad" p.ost~t©st mean" ©r fe) t© th© overall F¥ 
*» adjusted p0st-t@st iseaii*, th© ifedel appears'- ©JiXy a^erafe.- 
In short/ th# "adj us tiiints*' do wot' eoiipansate ©stirelf 
-for tha' initially ilbw scores. ^is,;aay be appropriate . 
and ^h®-esti{iat©& ef feet®- may Be unsMased. If we had 
coiiipi©€e-, faith in .©tir "adjustsisftts*' w# would fudge th© 
'-Hi^h/Seope ii^del t©-'ibe of ORly airferage .effectiveness, 
Th© results ©f the "..patched aiialyses, .- however, suggest 
that we nay hm **uiaaeradj|jstinf " « in both of tha "iiatohed, 
analyses t lite High/Scope W classes look somewhat better 
then their mtohed C©iisp4*i^o» elassejs. ' In th€"' five factor 
saffipie-analvses the:- differences are small but in the 
'.four faotor nateh' they are large and _ .statistically signi- 
"fieant. Our inclipat.ioii in this' ease is to equivocate— 
,^ the .High/Sc^p©-. ffipd©! ,may be .iisre effective than average - 
'but our data is not #t.r©n.g enough t© be convincing^, . 

The sit^aation for the REC Model is* also aiBbiguous. . 
^- Here om- basic problem is that thjsr® is only, on© PV site^ 
and -'no comparison' siteV The on^s sit©, however scores, 
in - th©' bottom twenty-five; pereent_o£_ th© sites in tmms ■, 
of, obs#rvad '^gair^s" gi"^i*ig tfe model th^-seeond lowest 
: ^ model ■ "observed" gains. .-S^hen contrast^ with the.. overall. 
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■pv ''adatisfisd pest-test" mean'^ and th® ©v^srali 
Coffiparis©!! adj listed pes test ®©aii*' .tli© KEC ^it© l©al£s 
abo^t" average., in th& mmtQhmd aaalfses, h©M©¥er, the^HC 
sit© soiiies »f£ leokiag feadly--s!i©winf 4if£©r©ne©s 
favor^mg the Coraparison f ^©lip ©'f ro^sglil]^ 0»3a,t© 0oS@ 
individual standard deviatioas* 'Siace there is ©nlf on© 
sit© Hen© ©f the soutw&Bts am statistioallf significant 
altheiigli thmf m^e clea'rXy omt ©f ^ fm ordinary = • |. 

: (4) ' "thm final two 3iiea©ls ClDG and Florida) emh 
show ©©nsistent patterns &f ; results » ■ All estiimtes for 

BBC sttfgest that it is soaewijat hmm ©ffaetiv# than thm. 

- • - ■« ^ 

' othm, m9,ae3.l . and' th& Cmpmieismymmd Sta^t f jtotps— the . _ 

diifemnces ar© all witMn the r Jig© -of -.47 to -.77 poii^ts* 
As vm noted earlier this ©ntige 0bsegve4 ©ffgct_seg^^ 
b© due to ofie oatXyinf BDC -^y site, - In this sit© the 
■ children ' aetually appe« t©'ljav© •'lost*' information 

(their average-, "gain" was , -0.24). Tw© things shoxild hm - 
'.noted abomt this sit©» First, ©f an original 85 children 
in th© site .oaly twentf were iftcludfed in the final- analysis 
sampl©— for whatever ' reason the reminder, were excluded 
. ■ (see Chaptar III £©r pospible reasons) . ■ This g iv#s iis 
...only an average of four children per class. Second, 
■. according to the OCD consultant this sita isnderwent great 
tariRoil Wring the sehool year.' The turmoil was perceived 
as having a substantial ©ffect on the teachers, advisory 
staff and .on the children. mk#n together thssm, two 
■factors sttgg@st tb «s- that the data ,£rom this site should' 
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.©f fefee EBC st^ielo • 

T!ie Florida psd@l also ^imm a fairlV^^^jai^*' 

'pattern ©f 'aefativ© results i^itli tL*s €ssg©epfci©a'. ®,f t-fe© 
'"^ofes^i-ved m^m'^ whmm it s^^mm t® de as ©ff siiffetlf 
feetfcer than its C©iRQaris©fi qmm. . In all of tim ©felier . 

• analyses* li©w©.Wc# eh© Florida P¥ iPoiel saefof to.fe© 

Coir^rison classes ^ ' la oslif on© insfeaace, • li©w®^er , dee^- . 
. -contrast 'd4f"f©f©ne©s lahUf e©nsi=steat, never c^eiei 

wry sli^hfc-ly-less @£fe@fciw thas& i&he otter _gl„^g§g 



ga^tfe^BQote '^P test,, mpu^h th@ flata-" ar®'. PQt . g#^S..:gg[^jgte 

fc© foe. eoia¥iiseliigf. .OirM..ffl€?d©l Ci4gM§ggE@l.-B§£A.a^. 

mitMGtiwe th&mh ^-amin we are onecgsvija|igM-»... , .l#--itogfe-^ 

■Vo Book 4a 



Plaiaii©^" ¥ariati®a -Sites rar&f© iii pre-Tp*?©' steans 
jprom 3«-l7 to 'S.Ol' points on the Book v4A ■. test .,iv%oef ^ly ' I ' ^ 
indiWa^iai stafiaard dmimti6nm. -ftof %4adl©; i: ill t^' gereeiife 
ranged fror^ pre-te'st mmm of Salt to" sit^lp^s tliaii 
• thirty.-three .pore^nt of an individual • ?ta^^iird dcviat-ienC 
Cqmparls©p.:'sitps "Isad pre- test seani ranging froia 4,14 to ,, 



9»44, , roiifhlf .lolS sfcanel^rd deviations.. The siiddl© 50% of . 
theteonparis©!! pt;e-test- means ranged ■ from 5.39 fcQu.&*18 or 
less than- tliirty-tteee p©rs©nt of a standard deviation* '. ■ ■ 
thus, itlioufls, it appears as if^th^re is a m-de range of van-.. ■ 
ation in pre-tdst iseans tfet bulk of tht sites faJJLin a very 

■"^served gains-' ISeMve im a s©«ewliat ''different way.- . 
^he rafege o£ P¥ faifcs.is from -0,14 to 7.8S>liiXe the range 
©'£ Comparisoii site '"©biserve^i faiilii" is froi»- -0»16 to '?«.94 , 
■points— eacH range* repraseoting rowghly 2.4 individual stan- " _ 
d'ard deviatiens-. fhe siiddle fiftr. ^©rf2©nt ©f P¥' «gains" 
raafes- from 2,30 to 5«08, -roufhly ©.9© standard deviations 
while the middl© fif t^' percent ri%fa_ ©r Coiaparisbn^^_»'gains** 
is from 2..^% to 3.S9# roufhly 0.6 standard deviations* .On 

, ... * • ■ :■ ^ ■ * 

'.the • swrfaee these differeBees stiff est that Idme Pf a.nd 
Cowparisen- sites may differ greatly^ in 'their effectiveness-, 
ill -impartijfig knowledge of. latfcers^ iitimba^rs and slmp^. names ^ 
■with the average PV sites appearing .slightly more e^cf©cfeivev 

"atbeiiuat© as tMf ' ^ for thQ ^ook 30 tisfe* Tha^ mod^l -with _^ 
thQ' lowest **faifi-iita'^ si-te- Is again EDt C^he.same site - . 



r^smlts 'f rop thi's site wxll ba^tsco"aiited. .-f oir' the reasons 
given earlier^ EDC also has the site with the sesond lar^st. 
■^gains'* though iE' shoiild .be- n&tmd that thq on-sit© .Oomparison 
classes for this site had th©, la^-gest av©.£agQ' gains of any -©C 



the^PV or Comparison sit^s*^ Qnly ona PV model (the Enablers) 
ijt^^ two sites in thm bottom twenty-^fivei percent of average 
V**gains^** Finally, pne;iRoael (Kansas) has each of Its two 
;sites in the top t|ienty-five percent of average'^^'^ains.'* • 

^ ^ - ■ \.: . ■ ■ . - ' 

B*. Model to Modal Pi £f erences 

- Table VII- 6 /shows model to -model differfshces in **gains*^ 
^for the PV mode^ls and their Comparison sites • The range in 
^observed* gainsr *^f©^ Modei:!^ is considerably larger ~ 

than for Book 3D The PV modal showing the smallest "gains" 
is Bank Street (1.88 points) while the largesit'^gains^are 
made by the Kansas model (6?. 06 points) * This is a difference 
of roughly 1*4 iridividuar standard deviations* A simila^ 
range exists when mtidel ^observed gains.** are contras tied 
€o their Comparison groups' "observed gains**' Here the range 
is. five points, from -0 • 85 points ^ favoring the Comparison 
grp\2p from Bmnk Street r to 4*19 points # favoring the 15ansas 
PV model* Thjree ©f the contrasts show statistically signi-* 
f leant r^sDJilts favoring /the RV gr'oups (the Arizona, 0. of 
Oregon and of Kansas models) • Similar resultis occur in 
the contrasts between PV and Gomp;4rison **observed-expected 
gains," Again th^ range is ro\ighly five points and the same 

■ ^ ' -''if' / 

three. gVrteode Is show favorable statistically significant 
results* In neither set of contrasts in Table VII76 is a 
Coiiipariaon set o£ classrooms significantly more effective than 
the PV model classrooms* 



Model Statistics for fch^ Book 4A Tasfc 



Coi\5mn X ijhovs th*:^ 'snean gai'n for children in the ctfedcli 

Coli^mn 2 ;shows thc'sneai^ gain for Comparison child^^^in in model 
• location* " : ■ 

CoXxsi^m 3 r,h0VB the diffarenco bctwoon Column 1 and Coima z* 
(h poiiiti'w n^oxo indicates that FV children oainod iRoro 
than Conjpariiion childrij'n) . r 

Colurrm 4 niimm thii difforcncc l^otwocn PV and Compariion children 
in ob^crvcd-o^jH Ctcd galnis, ^ 

*fhc* individual ii» tho unit of analycis, . ^ 



•* 

Model 


* W 

♦^G^iriia^i*' 


* 

Comparisqu 
"Gains" 


-JMams 
Comparison. 
*'Gains*' 


(obBcirvcd*expected) 

(pbsei^ved-cxpectod) 
f^ains** 


Far West 
I^aboratory 


S0«3,91 
3.60 


, . 3.15 


0.68 


(^86. 


Arisrona 


"4 ♦IS 
S*33 

132 


3. 71 

3.56 

61 






BanH St* 


3.72 

1.86 

117 


3. 70 

2.71 

94 






0* of 
Orctfon 


4 *<}6 

5*40 

'"■^'^ 182 


4.08 

168 


1.32*» 




0* of 


4,01 I 

6.06 

105 


i.39 

1.87 . 

61 


4.19^** 


.44,0*** 


High 


2.39 

121 


A. 12" " 

3.15 

96 


-0.76 ' 




U. of 
FJorldti 


4*66 

4»64 

110 


4.36 

4.72 


^0.08 


-0.03 


KDC 


4.58- 

4.25 

138 


5.19 

4.19 

123 


0.06 


0.#9 


U. of 


3.47 
3.48 . 

4 2 


3.42 

2.39 

31 


1.09 


1.23 


iu:c 


• 3.60 
(2.71 

4^ 










3.19 

2. SO 

115 









* statistically significant at the .05 level 
Statistically significant at the .Dl level 
*** statistically significant at the .001 level 

i All children m the basic analysis sample were used 



(see Chapter XII) 



C. ' "Adjusted Differences Between Groups" 

The patterns of results for the BooJc 4A test are noii- ' 
sidexably clearer than for the Book 3D test. Of the 108 
contrasts roajae in Tables VJ 1-7 and VI I- 8 ^ twenty are sta- 
tistically s|Lgnificant. Teh of .the twenty statistically 
signif icant iiiffer^nces occur > for one model (U. of Kansas) ; 
the other teji statistically sifcrnificant diffe'rences are 
scattered ov$r four models. Four patterns stand out in the-. 

data. I ■ 

1) The University of Kansas model appears to Be cOn- 
sideralbly more effective than/the Comparison Olasses and than 
the other models in imparti/g information tested by the Book 
4A test. Th6 Kansas model, in every analysis / has the highest 
estimated "effect." . 1 1^' average "observed gain" is^ roughly 
0.75 points higher than the next nearest model. It exceeds ' 
its Comparison groups by over four points in both "observed" 
and "observed-expepted" gains. In the direct contrast betwefen 
models its "adjusted effect" exceeds the overjall PV mean by 
2,71 points. The smallest estiroated effects for this modej^ 
occur in the regression analyses contrasting PV model "adj^iisted 
means" with the overall pooled Comparison mean— these 
mated effects" are roughly 2.23 points. ,In the other analyses 
the range .of "estimated effects" is from 3.11 to 3.88 pc^ints. 
There seems to be little question but that the Kansas model %, 
was more effective than the average of the other models and *- 



tl^n the Coii^arisoni classes in 19?ip--71 for th^ Book 4A test.| 
The effect seeiris to be on the order of 0*70 to ly3 indivMual 
leyeX Standard deviations • ^ ^, 

/y 2) Both the University of Oregfon ainwi the University of |J 
Pittsburgh models "show positive "esi^imated effects** in all 
statistical contrasts • Though st^tl^f icplly significant in on:|^ 
a few instances/ the pattern Of effects^ together wi1:h the 
University of Kansas finding/ strongly suggest that the 
highly structured and academically oiriehted models are some- 
what more sui?cessful than the Comparibon classes and' than 
most of the other models in imparting to children knowledge 
of letters, numbers, and shape names. 1 

3) None of the other models consistently have positive 
'•estim.ated effects," though Far West a^d Arizona each exceed 
their comparison groups in the matched classroom analyses 

by a substantial. margin* The analyses in Table VJI-7 indicate, 
however, that Far West and Arizona hav^i only average effect- 
iv^ness. I ^ - 

4) Two models show moderately consistent patterns of 
negative "effects" (REC and Enablers) • Two of the Enabler 
sites had "observed gains" in the bottom quartile of site 
"gains." When contrasted with the overall PV mean, the 
"adjusted mean" for the Enabler model was roughly one point 
lower. In contrast to the overall Comparison group they wer^ 
significantly different, with an effect of roughly -1,4 points 



" . BOOK 4^ 

Kodel •effect** estimtes for the tesit* Columns 1-4 fhonf differences 
between "adjusted** PV md^l loeans and «oine stemdard.' Colwn 1 showa 

the simple contra»fc» between the pv rsodel ^ad;Jiiated^'iti^top and an un-* ^ 

veighted grand naean ©f the model means for an exact least squares ond 
way liNCOVA* Columns 2 and 3 show regresition coaffficients for .e^ch ' 
modal in an analysis where all of the <?ojRpari»orf cXasses a^e pooled 
together to form a comparison ♦*model*^* f he reglession coefficients can 
be thought of as representing the difference bejbwisn the ^adjusted*' PV 
model means and the '^ad justed** Con^arison **model** me^ans. Column 2 shows 
the coefficients for a regression anclysic not 'allowing for separate 
slope coefficients for the covariates for the different models. Column , 
3 i&hows i:he coefficients allowing for separate model coefficients for 
-tho. PSi pre-te$it and for perceht prior preichool^ Column 4 shov;s the^ 
d:i,fferoncc between PV and Comj^arison group '*adiiisted'' means within 
models for sites with both a PV and a Comparison ^giroup. The estimateii 
are 1 degree of freedom^contrasts in the framewdlrlc of a one way JUiCOVA 
des^ign* Column 5 shows the PV and Cojs^arisoh n^s for column 4 analysis* 
A not^e following the Table list^. the covariates used in the analysis. «s 
In aliv analyses the classroont is the unit of analysis* See text 
(Chapt<iirs V and VII) for further discussion of the approaches.. \ 



A 

Model \ 


. Kstim.effects 
' aroiind PV un- 
weighted mean 


Estimated effects 
of PV models 
against pooled . 

compar* classes?' 


convrasv 
PV V. site 
comp. pooled 
models^ 


i 

1 

PV Cou^p. 


analysis I 


»naiv^i«a 2 


Far West V 
Laboratory , 


n»8 


-O.is 


-0.51 




4 2 j 


Arizona 


8 • 


0.13 


0.03 


-.28 


4 4 1 


Bank St. 


0,^4 

11 


0.09 


1*10 


-.06 


11 8 j 


U, of 
Oreaon 


1.76* 

12 


0.S9 


1.07 


0.95 


12 12 j 


U. of 

Kan/ian 


8 


2.22** 


2.24** 


3.11** 


8 6 ! 


lli tjh - 

^*^A*opf> 


^2,40*** 
12 


•0.92. 


-0.84 


-0.47 


12 9 j 


U. of 
FlorS d*t 


-0*67 

11 


-0.?3 


-1.26 


-0.63 


11 11 ] 


Pittuburoh 


-0.17 

n 


-0.00 


-0.08 


-1.22 


11 9 i 


a. 21 

4 


1.75 


1^77 


1.45 


4 4 1 


mc 


-1.37 


^ 1.10 


•1.42 






nn«i}>lcrs 


-0.92 

- 12 


-1.69** 


-1.37* 




! 


Grand Mean 


9.23 


9.23 


9.23 


9.25 


i 

1 



TABLE VIl-7 
(Page 2) 



* ' Statistically significant at the ,.05 level , 
.** statistically significant at the ,01 level • 
*** Statistically significant at the i 001 level / 

1/ Only PV classroQins are included in this analysis, 
The multivariate P with the PSI, Book 3D, and Book 4A ' 
in. the analysis is 2.36,* significant at the , 001 level . 
The overall multivariate P for Book 4A is 3.70, signi- 
ficant at the .001 level. . ' 

2. Both analyses were in the regression framework v;ith ■■ 
the pooled comparison classrooms as the "dummy variable" 
left out of the regression. Analysis 1 did not contain, 
separate slope coefficients for the various models. , 
Analysis 2 allowed for' separate slope coefficients for 
PSI pre-score and prior, preschool experience. .Analysis 

1 explained 70.6% of the total variation y analysis 2 
explained 73.4% of the total variation. 

3. Only sites with both PV and C^Sipai^on classrooms 
.4on or off -site) .were included in this analysis. 

Note: All analyses included the|following' cdvariables: 
PSI pre-test mean, Book 3D pre-test mean, Book 4A pre- 
test mean, mean age, percent black, percent Mexican- 
American, percent female, mean income, mean household 
size, teacher experience in Head Start, teacher certifi- 
cation, mean mother ' s education , percent prior preschool, 
average staff working conditions ,• whether Tthe site is El , 
or Ek. In the analyses in colxMnn 1 the Variable "site 
administered by CAP or by Public School "\ was also included. 
In /the regress;ion analyses in columns 2 knd 3 teacher 
reace was included. In analyses of the atanforf-Binet, 
the Stanford-Binet pre-test was also incl\uded as a covariate 
these, analyses used only Level III sites. .In analyses 
of the Motor Inhibition only classrooms with valid Motor 
Inhibition scores for both .fall and spring were included. 



Select<gd ^feaW«fcio«^ fosf HatcJiesd eia^si^oom Analyula of Book 4A 
for the 5 factor Katch 



CS^e Chaptiar V foir aasioifiption of mtcning |»roc«dtiM«0 
CoXtimn 1 «how« the number of mtchoft pairs of oltpirooifti for 
tho woael. Colmn t shows tho covariate wsfmi for twh model 
(PV pre-tcfit^ Matched Compariuoii pro-test) . Column 3 shows 
the unad:^ufititd dcjpendant Variable means for esch »od«l (PV 
p0at-to2Sft * Matched Compaifisonjpost-^t^jat)* Columns 4# 5 ind 
« show adjusted dependent variiblea for each model (febo OV 
adju&ted for tha covariato) under three conditions of esti- 
mates of the reliability of She covariafee (column 3 estiiiMites 
the reliability as 1*00, column 4 as 0*00 and column 5 as 
0.60)* !Phe tord-Potter correction is used to "correct* the 
covariate for itj; jwaliabiiity* 





1 id^varxate 
. j Heari 

. Coitip* Pre^ 
il*sl test 


Unad*ius^tied \ 
Difference 
PV Post-*Tesi 
>- Comp* 
post-^i^est 


'Adjt 
(PV Post-T< 
{Ad:j|usted^i 
covafiate 
Itel. « ItOO 


iste4 Oif f eri 
tst Cdwp. 1 
foif Pre*»Test 
Covariate 
Eel. •» 0*80 


ponces 

post-l*e»t) 

jpovat;ia.!1?J^i 
"^covariate 
ftel* » 0.6(? 


Far W0s*t 
Laboratory^ 


S 


0.S4 


O.jBl 


0.S2 


0,82 


0.82 


Arizona 8 


-0.98 


0.44 


" / Otl4. 


0*44 


0.43 


0anjk St. 


11 


0*03 


-0*10 


0.10 


O.U ^ 


0.11 


Oregon 


12 


-0.19 


0»97 


^ 0.97 


0.91 / 


0.97 v 


Univ, of J Q 
Kansas t 




1.54 


3,23^** 


3.23*** 


3,23*** 


l^^K 12 


-0*06 


0.17 


0.17 


0.17 


^.17 


oniv* oi 
Florida 




-0.32 


-0*24 


-0*24 


-0.24 


, -0*24 




11 


0.13 


0.S7 


0.58 


0.58 


0»58 


uiTiv. of 
l»ittiihurc|h 


1 4 


1,31 . 


t.n 


2.77* 


2.77* 


2.78* 


KMC j * 


-0/45 


-2.50 


-2,58* 


-2.58* 


-2.59* 


Enablcrs ; 

♦* 


0*24 


-0.62 


. -0.62. 


—0,61 


-0*61 



• statistically significant at t'he ♦OS level 
** Statistically significant at the •Ol level 
♦♦♦Statistically significant at the .001,;ievel 



The overall cotrelation between PV prer^ and Comparison pro-test 
matched classroom measures ^ .51* rrl^^ overall F for the test of 
homogeneity of the covariate regressji^^n coefficient? » 0*7 0 , 
2'rhe rcgrcsaion cocffi^piGnt for the .ci^variate for the analysis with 
reliability (r^^) estiinatod as l.Od « - 0.00? with r^^ cstimatod as 
0/80 the coo£ficient ^ >-a01? ^or r^^ « 0.^0, the coefficient « 01 ,. 



\ 

\ 
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Sel^ptad Statistics for Hat4*^ cimnxoom hmly^lB 0k Book 4h 
tot tM 4 fabt0x l^ttsh \ 

(S<ie Chapt&r V for description rmtchingf proceduxei,) \ 
eol^ymii X shws the nuttiber <jf miitchad pairs of clMstoom^ fw 
th«i iw>ael» coiumi 2 shows thift <J0v^riate ni«im« for Mch mc^l 
(PV prc-t<asfc - Matched Co^nxtson pro-tott) » Column 3 »howk 
the unadjusted dependent V^riahlo mmn for each model (Pv V 
pOfit-iasit - Matched C^piitlson po»fc-'tosu) • Column* 4/ 5 *n^* 
« i^how adjusted dependent vari^blc^ for each model {the DV v 
ad4u&t^d for the covarlat^} under thrae J?onditiott» of e»tl- \ 
watea of the reliability of the covariate tcoXumn 3ve»timate4 
the reliability ar. l.<50^ Column 4 as 0*»O and column 5 «» 1 
0.^0) . 1*hc tord-S?art^r cfcirrection ia uied to ^•correct'^ the ^ 
covariate for its reliability* 



{ covariate 
I Hoan 

Comp, Pre* 



Tei^t 



mndjwjtu^f "Adjuated pifference* ' 

Difference | (PV Post-teat - Comp* >Fo»t-Tes^) 

PV post-^estL^dJatt«.4^f^^^^ 
- Comp. pcovariate ^ ^ ^ ^ 

Poat-1?e»t jKel* * 1*00 



Covariate Covariate 
Rol- * 0*80 mi* « 0.6C 



Far west j 
l^aboratory 



1.2? 



2*01 



l.ia 



0*90 



0-53 



Arissona 



0.75 



1*48 



l.«6 



If 97 



Bank St* 



11 



0*26 



-li.05 



-1*25 



-1*29 



•'1, 37 



ilniv# of 
Oregon 



12 



0*24 



0,56 



0*30 



a. 34 



Univ* of I g 
Kansas S 



"0*06 



3*80 



3*85*** 



3*86*** 



0*27 



3*88$** 



gcoge^ 



UnivT'oT' 
PJorida 



12 



0*37 



0*88 



0 . 80 



0*54 



11 



-1*20 



-I as 



-0*23 



-0*01 



0*12 
"0*"3F 



11 



-0*50 



-0.84 



-0*47 



-0*38 



Univ. of I 4 
Pittg^burgh; 



"0*1!> 



1*09 



1*23 



1*27 



-2*07 



«0*$4 



-0.85 



-0*22 



1*32 



-0-18 



-0*38 



-1.07 



^1*24 



-1.53 



* Statistically significant at the *05 level ^ 
Statistically significant at the ,01 level ^ 
***St«itistically significant at the *001 level 

^Thc overall correlation -between pre- and Comparison pre-test 
matched classroom measures -0*1$. The ovcrall-F for the test of 
homogeneity of. the covariate regression coefficient « SUStl- 

^Yhc roistau^ion coolficicnt for the- covariate for the analysis with 
reliability (r^* ) of.timaiea nn 1*00 <K73 with r^i estimated a^ 
^•80 the cocmcient for r^^ n 0.00% the coefficient ^ 1,21 * 



1 «^ t i 2 



or roughly O.SCf' standard deviati<^ns. In contrast 
matched Comparison samples th^ir "effects* ranged i|bm -0.60 
to -1.53. Though only two o^ fche contrasts were s||lilficant, 
the overall pattern does suggest that the Eiiablfer libdel'* 




was upt as effective as the other Head Start progr;^ as 



assessed by the Book 4A test* . 

The results for the REC model are less clear The 

...'■< ' . X '•■''11 '■ . 

lack of replication for this model again makes it#. effects 

difficult to assess. Ghi'ldren in "thir model "gaM^d" 2.71 

points on the average on the BobJc 4 A test, placii^ .them third 

from the bottom in terras of mean model gains. Compared to 

the overall PV mean/ the adjusted REC "effect" ife -1.37 points 

while contrasted to the overall Comparison adjustj^d mean, the 

ef fect was positive (roughly 1.4 points) . In the two matched 

classroom analyses it had negative "estimated differences." 

In the five factor sample' analysis the negative differences 

are considerable (about 2.^58 points or roughly 0.80 standard | 

deviations). The four factor sample analyses yield much 

smaller negative differences. As in the case of the Book 3D 

test, we are ambiv .ieht about reaching conclusions about REC^ 

since it has only one site. 

in stiiwnary , there is a clearly exemplary model with 
respect to the Book 4A test. The University of Kansas model 
exceeds all other models and the Comparison classes in all 
analyses by a substantial margin. Moreover, there appears to 



. be soim evidence that the other two highly structur@a and 
— ■ — tr^ — — — — ~— : v:,, ^. 

academically oriented models (University of Orfe gon and Oni- 

varsity of Pittsburgh) are aJ.so especially effect ive on this 

outcoroe measure. There is some evideftce that the Bnabler 

model and the KEC model are ziot as effective as the other 

• Head Start programsV but the evidence is not at all conclusive. 

VI. The Preschool Inventory 

' A. Site to Site Differences (See TaBte~VlM) 

Differences among PV sites in PSI pre*score mean^^^ 
from 23.71 to 49.05 points, roughly 2.5 standard deviations. 

The middle fifty percent, however, range from only 29.98 
to 37.77 points or about 0.80 individual standard deviations. 
Comparison site pre^test m^ans are somewhat more closely 
bunched, ranging from 26.49 to 45. J5 points, about 1.9 
standard deviations, whj^' the middle fifty percent range 
from 30.80 to 39.60 poi\ts, or 0.90 standard deviations. 
, T^iese initial differ^ces antong the sites are comparable 

to the size of the differences; found foSr the Book 4A test. , 
The differences in relative "gains," however, are consideicably 
smaller. For the PV sites the range of "observed- gains is - 
from 2.08 to 17.0 points, roughly 1.4 vstandard deviations 'of ^ 
the • individual test scores. This contrasts with a difference 
of 2.4 stanviard deviations for the Book 4ft test. Comparison 
site "gains" have a similar rang^'^'^rom 6.72 to 19.07 points, 
1.3 individual standard deviations .The middle fifty ^percent 
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of the PV ^it^ gaias r#ng^ from t> 15 to ,12 •S6 |foiats— • 
, a difference . of «iiGer' 0.40 standard -cleyiatioks . ».*l?tte : 
roididie fifty percent of Comparison site,- "faissaf :raage / 
from' 10. 28-. to 12. 4S points,, ssnder ,p.3S. standara a©via-/ 
tions, .Clearly^ the majoritv ®£ sites la this stiidy ' are 
: Closely bunched in terms of "fains" on the PSI test. ■ ■ ■ 
None of the PV models has more thaii-. one -site;, in the' 
..bottom tvi'eaty-fi^e .percent of _ the distribution of PS-I 

"gains" .though '-the': Enabler _mpdel has --Qne site at the, - ■ 

. • ... .. 

-\ 25th pereenti.le aisd one below, .'fhe-.third ;E»aMer_'?ite 

.ihowever, has a "gain** of l$^m points,, placing it.'' ,; 

seco-nd in *' observed gains'^* ammg sites ^ . MQxmovmT-^ thm. 

xmd^l CQ'ntmining tho site witfe,^;tte^^ail0st ^gMn^ -^^ 

is also tbm m^dml with the -si'te * having, tha l-argasl 

^'gaim*". ' ^Hla model (ni^h/BM^mf-h^ff''tMo^^ ' 

tte 75th p#.reei^til# in **g'alns"^'V. jThis h^pp^nB- tpr ■ m ^ ' 

other .modal* " \ ■ " . , . 

II. ""itQdel to Med© I Difference® ' ^ .... ' ■ . 

fsbie VII-U displays; Bodei- to-nodel dMffreneisg in 
P¥ and Coroparisoin "'gains*'" on,th# PSi^\ For the PV" models 
thm *'obs#rve>d gains"*" range from 9.19. to 1 3 1 0 points. 
roughly 0*40 .standard daviat.ioris« 

, Whan Py and Comparison olassas withii^ models arm '^ 
compared on observed PSl gaias the' dif f erene^s ranf ^ fr©ii 
I. €4 points favoring the Univ^trsity of Pittsb^irgh mod^l 
to *-5»24 points favoring th,^ eompmrison group for thm Par 
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West model. It we ©lissifjat© th© Far West moa©! froist our 
consideration becaus© of inappropri-^ite (m-sit© con^arisons 
the range if halved # going from 1.64 to -2.3-S <tte latter.. 

'dif f«r-.';;ifiqes "^£© essentiallf similar to th© '^observed" . 
dif fisrenees. ■ . . ' 

TheKi as s©f?ie indication fsom fable VII^s-lO that the 
F#r West' LaboratdE-f, the Itoi^ersit^ ©f prefon a«a the ■ 
ICC models ar© soMe^ifhat. less tiiaa " average in their 
effectiveness*^ -assessed by thm PBl. The Far W©sfc and 
the EBC »9d©ls "hav© the' siaallest model '""fains" , botli being 
siaaller tisan any of the Corapirisop/grdtAp'* fains." mile, 
tlje Oniversitf of Or©f©Ji P¥ msael has a *'fain*' in the- 
average ranfe, its Comparison gmup has the seeond . 
•largest avaraf© ''^faiffi*' mw&nq. the Comparison graaps. 

Co ^'Mjasted Biifmmncms fms^m Greiaps'' 

©f the 108 eoatragts in Tables ¥1I~13. and ¥U-'12 
only nine r©aeli statistical sifnificance,. Moreover, 
ther© are m ci©ar patterns of results as there were for 
tlie Bejok 4k test, Th© lack of interesting reswits can 
j»st clearly tee' seer* hf looking at the matched elassroom' 
re'smlts in Table i.2c Of the 66 eontraets' in this Table 
only OHO maolms statistical. sigmifieaBCf^ ana that only 
%ih©ii the reliability ©f the comriate is assTOed to 
©nly Oo$0=='»ufi£ioyi5te«ily an andercstimate Cseo Chapter ¥1)'. 
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Hodtel ^^ffeet* estirmtos, for the testr* Coliujais l'*4 show* differences 
bc^ttNKien ** adjusted'' PV R>odel ?aeasisi and soTdB standards Column 1 shows 
the Bimpl^ contrasts b^i^een tho PV Biodel "adjusted* means and an un- 
iMei^hted qmnd tmm of the mdel laeains for an ejcacfc least squas^a one 
nay M9C0VA« Colx3isa:is 2 and 3 eho?? regression coof ficienta for each 
Btodel in an analysis %ihera «ill of the coii^ariion claaaea ara pooled 
t09<ither to fom a cor?^ari«on *?rtodel*. thio rcgrcsaion* coefficients can 
be thoii^ht of as representing the difference between the ^^adjusted** PV 
iisodel mmB and the '^adjusted*' Comparison •'sJodeA* meana. Coluim 2 shows 
the ^focffi^cienta for a regtesaion ar^alysis not allowing for separate 
islopei coc^fficicnts for the covariates far the different models. Column 
3 shows the coefficients alleging for separate model coefficients for 
the PSI pre-test and for percent prior preschool. Coluisn 4 shows the 
difference between FV and Corsparison group ''adjuated*' means within 
wodmln tor nltm with both a PV and a CoRipalrisott group. The estimates 
are I degree of freec|om contrasts in the framework of a one way MCOVA 
design* Column 5 shcw^ the PV 4JUE»d Con#arison n*s for coluwn 4 analysis « 
h note following ^e ^iible lists the covariates uaed in the analysis ♦ 
In 1.11 analyses the classroom is the unit of analysis « See text 
(Chapters V md VIIJ for further discussion of the a|^roaches» 
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•^2*05* 
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11 


*2*54* 


-2.49* 
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11 9 J 
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0.93 
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1*51 
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i. ^ , 


-0.37 


-i;2*6 
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Cr^nd mm 




46.14 


46.34 


46.56 





TABLE VII^ll 
(Page 2) 



* Statistically significant at the .05 level 
Statistically significant at the .01 level 
*** Statistically significant at the .001 level 



1. Only PV classrooms are included in this analysis. The 
multivariate F with the PSI, Book 3D and Book 4A in the 
analysis is 2.36; significant at the .001 level. The 
overall univariate P for the PSI is 2.27, significant at 
the .05 level. 

2. Both analyses were in the regression framework with 
the pooled Comparison classrooins as the "dummy variable" 
left out of the regression. Analysis 1 did not contain 
separate slope coefficients for the various models. 
Analysis 2 allowed for separate slope coefficients for PSI 
pre--score and Prior Preschool Experience. Analysis 1 
explained 78.1% of the total variation; analysis 2 ex-- 
plained 81.4% of the total variation. 

3. Only sites with both PV and Comparison classrpoms 
(on or off-site) wer^ included :>n this analysis. 



Note: All analyses included the following covariables: 
PSI pre-test mean ^ Book 3D pre -test mean, Book 4A pre- 
.test mean, m^an age, percent black, percent Mexican- 
American, percent female, mean income, mean household size, 
teacher experience in Head Start, teacher certification, 
mean mother's education, percent prior preschool, average 
staff working conditions, whether the site is El or Ek. 
In the analyses in column InJ^J^s?^ variable "site administered 
by CAP or by Public School" was also included. In the 
regression- analyses in columns 2 and 3 teacher race was 
included. In analyses of the Stanford-Binet , the Stanford- 
Binet pre-test was also included as a covariate— these 
analyses used only Level III sites. In analyses of the 
Motor Inhibition only classroom^ with vali^ Motor Inhibi- 
tion Scores for both fall and spring were included. 



TABLE VXI-12A 



Selected Statistics for Matched Classroom Analysis of the PSI 
for the 5 Factor Match 

(See Chapter V for description of matching. procedures*) 
Column 1 shows the number of matched pairs of classrooms for 
the model. Column 2 shows the covari4«|te means for each model 
(PV pre-test - Matched Comparison pre-test) .' Col -^mn 3 shows ' 
the unadjusted dependent variable ?neans for each model (PV 
po?3t-test - Matched Comparison post-test). Columns 4^ 5 and 
6 show adjusted dependent variable^ for each jnodel (the DV 
adjusted for the covariate) under thre^ conditions of esti- 
mates of the reliability of the covariate (column 3 estimates 
the reliability as 1.00, column 4 as 0.80 and column 5 as 
0.60). The Lord-Porter correction is used to **correct" the 
covariate for its reliability. 





j Covariate 

Mean 
W Pre-Test 
^ - Comp. Pre- 
N • s! Tes^ 


Unadjusted 
Difference 
PV Post-Test 

- Comp. 
Post-Test* 


"Adjx 
(PV Post-Tc 
(Ad j us ted J 
Covariate 
Rel. » 1*06 


isted Differ 
*st - Comp. 
:or Pre-Test 

Covariate 
Rel, * 0.80 


ences" 
Post -Test) 

CovarianceJ 
Covariate 

Rel. •« O.cC 


Far West | g 
Laboratorv' 


/ 1.09 


-1.18 


-1.76 


-1.90 


-2.15 


Ari>5ona 




-0.02 


2.57 * 


2 . 58 


2.58 


2.58 


Hank St. 




-0.75 


-1.41 


-1.02 


-0.92 


-0.75 


Univ. of j ,2 
Ororjon ! 


1.55 




0.04 


-0*17 


-0.51 


Univ. of j Q 
Kansas; 1 


4.71 


3.71 


1.20 


0.57 


* -0.47 


nigh 
Scope 


12 


2.04 


2.12 


1.03 


0.76 


0.31 


Univ. of 
Florida 


i 


0.20 


^0.35 


-0.46 


-0.48 


-0.53 


KDC 


11 


-0.64 


-0.54 


-0.20^ 


-0.11 ^ 


0.03 


Univ. of J ^ 
Pittsburgh* 


0.92 


1.39 


0.90 


0.78 


/b.57 


REC 1 4 


-0.58 


-2.82 


-2.51 


-2*43 


-2.30 


Enablers i 12 


1.62 


1.14 ' 


0.28 i 


, 0.06 


-0.30 



♦ statistically significant at the .05 "level 

*♦ Statifjtically significant at the .01 level \ 

***f>taListically significant at the .001 level 



TJic overall correlation between PV pre- and Comparison pre-test 
matched classroom measures « 0*82 The overall F for the test of 
homogeneity of the covariate regression coefficient * 6.7$ . 

^The regression coefficient for the covariate for the analysis with' * 
reliability (r^^l) estimated as 1.00 = 0.53 ; with r^.^ ostinjatod as 
O.UO the cQcfficient s .67; for r^t *= 6.60/ the coefficient « .89. . 



TABLE VXI-*12B 

Selected Statistics for Matched Classroom Analysis of tho PSX 
for the 4 Factor Match 

(See Chapter V for description of matcliing procedures.) 
Column 1 shows the number of matched pairs of classrooms, for 
the models Column 2 shows the covariate means for. each model 
(PV pre-tegt - Matched Comparison pre-teat) . Column 3 shows 
the unadjusted dependent varial^le means for each model (PV 
post-test - Matched Comparison post-test). Columnis 4, 5 and • 
6 show adjusted dependent variables for each model (the DV 
adjusted for the covariate) under three conditions of esti- 
mates of the reliability of the covariate (column 3. estimates 
the reliability as 1.00, column 4 as 0.80 and column 5 as 
0.60). The Lord-Porter correction is used to ^correct" the 
covariate for its reliability* 
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V^v 1 n _ ^ ffvt — — M.. 

PV Pre-Tcst 
- Comp. Pre- 
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unao^ustea 
Difference 
PV Post-Test 

- -Comp* 
Post-Test 


''Adjusted Pif ferehces" 
(PV Post-Test ^ Comp. Post-Tcsw) 
(Adjusted fog Pre-Test, leoyari&nce} 


Covariate 
Rel. - I.OO 
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Rel. ^« 0.80 


jjCovariate 
fRel. » 0.6C 


Far Wc£;t 
Laboratory 


8 


5.03 ^ 


2.67 


-1.09 


*2.03 ^ 


! -3.60 


Arizona 




. 1.39 


0.75 


-0.28 


-0.54 / 


0.98 


Dank St. . 




-2.39 


-4.11 


-2.31 


-1.87 1 


1 -1.12 
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Oniv. of 
Oregon 


12 


1.59 


0.72 


-0.48 


-0.77 / 


j -1.27 


Oniv. of 
Kansas 


6 


6.12 


4.75 


0.16 


-0.98 y 


1 -2.90 


High 
Scope 


12 


4.43\ 


5.78 


2.46 


1.63^^ i 


0.25 


Univ* of 
Florida 


11 


-2.51 


-3*67 


-1.78 


-1.3(1, \ 


-0.52 


EDC 


11 


-2.25 


-1.77 


-0.08 


0/34 ' 


1.04 


Univ. of 
Pittsburgh 


4 


-4.14 


-5.71 


-2.60 


-1.83 


-0.54 


REC 


4 


-4^45 


-4*56/ 


-1.22 


^0.39 


1.01 


Knablcrs 




4.74 




-1.33 


-2.21 


-3.69* 



* .Statistically , significant at the .05^ 1^ I 
*♦ Statiiiticaliy significant at the .01 level [i 
***;>taLictically significant at the ^001 level . 

, overall correlation between PV pre- and Comparison pre-test 

\ matched classroom measures « 0.2 ia. The^ overall F for the test of 

\ homogeneity of the covariate regression coefficient » 2.08 . 
Tho regression coefficient for the covariate for the analysis with 
^liability (r^.^) estimated as 1. 00 = 0.75 ; with r^^ estimated as 
o\86 the coefficient « 0 .94 ; for r^t = 0*^60/ the coefficient = 1.25 . 
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Three rough patterns can be suggested , however. 

(1) Firsts there is no indication of special effect- 
iveness for the academically oij-iented, highly structured 
models on PSI gains. . Although Pittsburgh does have the 
highest model "gain" its contrasts with other models and 
Comparison classrooms suggests that, ais a model/ it is 

of only average effectiveness. Similarly- both the Kansas 
and Oregon models show only average effects. 

(2) The Arizona, High/Scope, REG and Enablera models 
also show inconsistent and generally average results. 

* While these models appear similar when their sites are 
aggregated they are quite different when individual sites 
are examined. For example, both Arizona sites have 
observed gains in the middle range of site gains while 
the High/Scope sites show great variance in their observed 
gains, as do the Enabler sites. 

(3) Far Wes^ Laboratories, Bank Street, the Univer- 
sity of Florida, a^hd Ebc show generally negative estimates 
of effects though few 6f the contrasts are significant 
and oqgasionally even of a positive sign. Both: EDC and 
the University of Florida show Stat : ?:ically significant 
negative estimates when contrasted the other PV models 
and to the overall pooled Comparison classes. Far West 
and Bank Street do not show statistically significant re- 
suits though, with one exception for each model, all of 
their effects are in a negative direction. The, effects. 



-198- 



"howovo^T ~y YTT never exceeding 0.30 standard devi- 
ations when the Comparison group is appropriate. Our general 
^ conclusions, therefore, is to assume that t he models are all 

of roughly equal effectiveness. 

Th e lack of clear difVer-eiTces ^among^^mod^ in their 
effectiveness as assessed by the PS I may be due to the 
nature of the test. It, more than any of the other tests 
examined here, was designed to tap the gene ral dimensions * 
of a preschool experience. As such it should be less : 
sensitive than other tests like Book 4A to specific 
differences in curricula. One indication of this is the 
relatively small range of differences in observed gains 
among the sites. It could well be that this test is • 
inappropriate f6r an analysis of differences' among 
curricula. Although it may serve a general purpose in 
pointing out particularly weak or strong sit^s (note 
the High/Scope site differences) it perha^ps is better 
suited' for analyses of individual differences among 
types of children. The report "Cognitive Effects of 
Preschool Programs on Different types of Children" ^ 
explores this issue in detail. 

VII. The S'tanford-Binet . . 

A. Site to Site Differences ; 

Table shows site pre-test means and observed 

gains for both PV. and Comparison groups. Only the sixteen 
Level. iTl sites arc included in the analyses of the Stanford- 
^ ' Binet. This excludes the Enabler model ^nd reduces the 
Q maximum numbef- of siptes per model to two. PV pre-test 

' r "HI 22 4 




TABLE VII- 13 
Stanford-Binet . 



Pre-tcGt means and mean "gains\^ (post-tOBt mean - prc-test 
mean) by site for I?V and Comparison groups. Site moans are 
unweighted averages of classroom means. . . 
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means range from 77,4 to 99,76 points— a range of roughly 
1,7 individual standard deviations in this sample or from 
a very low "normal'* level to the national average in terms | 
of national norms. The Comparison site range is almo«5t 
as large — from 82.52 to 101.80 points. The middle fi'fty 
percent of the PV sites , range from 87.62 to 94,95 points, 
a difference of roughly 0.5 standard deviations, while the 

middle fifty percent for the Comparison group range from 

. * ^ 

86.80 to 96.80 points. Thus although there are^a few 
sites with very low pre-test means, by and large, the 
sites cluster between one-third and one standard devia- 
tion below the national mean. 

Observed gains for the PV sites range froni -1.53 to 
30.59 points. The latter, however/ is an extreme outlyer — - 
without it the range is reduced to -l.§T'toT^2.05 points, 
a gap of about one standard deviation. The Comparison 
site range of observed gains is not even as large as v 
this reduced range, going from -0.47 to 7.73 points. Wh6^n. 
we lop3c^ at the middle fifty percent range of gains the 
PV spread becomes only about 4.6 points, from 2.70 to 
7.33 points while the Comparison site spread is also 
about 4.6 points, ranging from 1.20 to 5.66 points--roughly 
0%35 standard deviations. 

When we look at sit^s within models the. spread does 
not reduce quite so much as it does for the other tests. 
One model (Bank Street) has two PV sites in the bottom 



quartile while another (High/Scope) has the two sites 
making the greatest -gains. Of note, however, ia the fact 
that the Comparison sites for Bank Street also gain very 
little (relative to the other sites) while the Comparison 
sites for High/Scope are both in the top quartile of 
Comparison site gains. 

B. Model to Model Differences ; 

The spread in model to model "gains "is shown clearly 
in Table VII-^14, The High/Scope PV model far outgains 
any of the other PV models, averaging 23.4 points in "gains" 
while Bank Street lags behind with an average ^'gain" of 
-1.73 points.* Two other PV models show higher than 
average gains — both the University "Of Pittsburgh and REC 
show gains of slightly over eight points, -flfll of the 
other PV models ^gain between 2 . 5 and 5.24 points, a 
difference of less than 1/4 of a standard deviation 
of individual test scores. The Comparison groups show 
less variation with the Bank Street Comparison group 
having the smallest^^gains" (-0.65 points) and the High/ 
Scope Comparison group the largest (7.18 points). 
In tTle^cbnCxa 

expected" gains for the PV and Comparison groups the High/ 

*When interpreting these gains it is important to remember 
that we expect some deterioration in Stanford-Binet over 
the seven months a child is in preschool. Thus, all of . 
the models are producing slight positive effects (see 
Chapter IV) . r 



TABLn VIX-14 
Model Statistics for the Stanford-Binet 



Column 1 «howf$ tho lacah gain for PV children ih tha rnddcJ* 
Co3umn 2 uho^u the munn tjain for Coiuyuriuou ahildron in modol 

Culmn ti fihov/r; the dirfcrcuoo bctwocn Column 1 nnd Coluwn 2, 
(A, j#o:;itlvo j;coro indicatcis that ]»V children gninod moxo 
than Coraf^arii^ion children)* 
Column 4 nhow£> the dirCercnco betv/ccn PV and Comparison children 

. in obsorvod-expc^ctod gains, , 
The individual i& the unit of analysis. 
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36 
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2.49 
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9.34 
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5,24 

43 
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4.24 

51 
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3.43 
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0.70 


0.67 
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^A13 children in the* basic analysis sample were used 
(«K;o Chapter III) 
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Scope PV model stands out as clearly different from all 
of the others with an adivantage favoring the PV group 
of roughly 16.5 points. None of the other measured 
differences exceeds 3.5 points* Thus# in terms of 
simple gains and differences between ?V and Comparison 
groups there is only one main finding in this data — 
the High/Scope model appears to be extraordinarily 
effective in raisin<^jS^^irtford-Binet scores at least in 
the short run. Other than that there are no differences 
of note in the data shown in Table VIl-14. 

C . ** Adjusted Differences Between Groups '* :^ 

When the data in Tables VII-15 and VII-16 are examined, 
the picture becomes only slightly more complex* Six 
PV models require little attention; . Par West Laboratories ^ 
Arizona, Oregon, Kansas, Floirida and EDC all show small 
and inconsistent effects,. Note that this group includes 
two of the highly structured academically oriented models, 
an indication that this approach is not necessarily more 
effective than other approaches when the Stanford-Bine t 
is the outcome measure. In the following we consider the^ 
four remaining models. . ^ 

(1) The University bF pittsburg^h^^^m 
averaging over eight points. Although it appears less effective 
than the average PV model, (see Column 1, Table VII-^IS) it shows 
positive effects in all of the other contrasts, in Tables VI 1-15 
and VIl-16. Of the eight other contrasts five show 
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variable "site fi-dpiiiiist©ifea fer CUP or h'i Pulslie SeliQol" 
was also inelisded-o In the regression analyses c&lmmm 
.2 and 3 teacli#r raeis was inel^jfied, li?. aaalsrses.of t|i# 
Stanf<?-r^-Biifiet* fa© Scaufoifd-BiBe*' ©r-©-test , ?(tas also 
■included as a eovariat©— tliese aiiaifs©s Ms^d ©uly 
III Sites. ' In aiialyss©® of tte Motor mix bitioa ©Rif 
elassrooias with vaii^i Master laliibitioss seeres f©r ,^ 
'b©th fall ©pring w©if© iaelud^d. 



iBm Chapter t tor 4m^^^ip^im ®f mtiehing ^r^cedur©*^) _^ 
i til© m^fe^i? M mts^ed p^isrs of alassrcc^ 

^isa#34sai&€d vmi^l^ mmn for meh wmm. |FV 

$ d^^^ii^i^d tfa^mi^ist .mrial^l^s far cash Ctn© 
mdiw3tQd tor i^tie ewarra^isel tinier is^f^e^ ^©ndiitioi^ft of 

■€>A,i0r<> correct lis to ^^or^ct tiift ^ 





1^ c^war^feii^te ' 
































Bmk it a 1 ^ 




.-6* §2 




-S.44* 1 








! 














^^ol? ^ ■ 






















j ^ioM 










H— — 








1 .laSi 






Moll 




Worn*** 






^^^^^^^^i*"^^ > -" ^ — ^> 























** it.ttiicj-x.i^rif f=-4-?iiu^<^ tie-art .^^ tte «§1 i^^^i 



t>;'^jf^%'j|'V;>^.«'ii3^ yf- v^m <^:^^'^i4ti^^t® rC^i!u^uim% mpiti^lmnt * §sJi.c. 



Selected Staieisticjis for Mafcched Classrwi^ Analysis of the 
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CSe<i Chapter v.for deBcripfeion of matching procedums.) 

Coluinsn X sho%is thQ m^sb'Q^ ■■■t jsatashed pairs ot classroom* for 

the J^del* Colusswa 2 showra the covariat© mans for each roodel 
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pb©t-*t©g>t - Matched 'Comparison post-tc3t). Columns 4-# . S and 

$ »ho*i adjiust^d dapendi^nt variables for each im^dcl (the DV 
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0<£OK' '^Ite Lord-l*©rter correction is used to ^correct** the \ 

c^variafce for ita reliability* 
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statistically significant results— no difference is less 
than 5.6 points. or roughly one-half a standard deviation. 
Although it is difficult to draw a conclusion about 
Pittsburgh since it has only one site, there is' a strong 
indication that it is more effective than the Comparison 
Head. Start programs in imparting gains on the Stanford- 
Binet. 

(2) The REC model showed an average gain of slightly 
over seven points. When placed in an analysis directly 
contrasting PV models 'the REC model shows a highly 
significant effect of 17.20 points. In contrast with 
the overall Comparison group it also shows a statistically 
significant effect of 10.97 points. In the matched sample 
analyses, however, the REC model does not show a large 
effect (it never exceeds 4.65 points) although in all 
instances the .direction of the effect is positive. 1 
jWe are unclear about the cause of ti>is rather dramatic 
I set of differences between estimated effects. from dif- 
' f erent analyses . Prestimably , it has something to do 
with the form and nature of the covariates used in the 
analyses. Whatever the reason, however, there is clearly 
an indication that the REC model may be more effective 
than most PV and Comparison Head Start programs. It is 
important r though, to remember that there is only one REC 
site so it is impossible for us to reach a firm conclusion 
about the model. 



(3) The Bank Street model had the smallest "gains" 
on the Stanford-Binet of any of tl^e PV models. Both 
Bank Street sites had similar pre-test and gain -scores. 
In the analyses described in Tables VII-15 and VII-16 
Bank Street shows a consistently negative effect, ranging- 
from -11.33. to -3.13 points. Of <the nine contrasts five 
are statistically significant. / If we disregard the largest 
negative effect because it ocpurs in the PV model to model con- 
trasts, (the analysis we hav/ least confidence in) the range is from 
roughly -7.5 to -3.13 points. The only reason'we have not to con- 
clude that the model is less effective than the other ^^bde Is stems 
from the extraordinarily high pre-test means in both of its 
sites for PV and Comparison groups. Although these sites 
show moderately high pre-test means for the other outcome 
measures , they do not come "close to approximating the 
relative magnitudes of the Stanford-Binet pre-scores. 
This suggests that the Bank Street site pre-scores for 
both py and Comparison children may be over-inflated — 
perhaps because of an over-zealous tester. This would 
give the children: in these sites little opportunity to show 
impressive gains on the Stanford-Binet. A somewhat more 
conventional interpretation might be that a regression 
art.ifact was working on the Bank Street scores-^-the high 
initial scores would have to be due to substantial positive 
errbrs in both sites for this explanation to work* Since 
the average classroom N for the Stanford-Binet analyses 
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is only about five this explanation may be plausible. Our 
inclination, then, is not to reach a firm conclusion about 
the Bank Street model's effectiveness for the Stanford- 
Binet as the outcome measure. 

(4) The High/Scope model shows dramatic positive 
effects in all analyses of the Stanford-Binet Estimates 
of adjusted differences range from 10 to 17 points — 
from 0.7; to 1.3 standard deviations. This effect is • 
comparable in magnitude to the effect found for the 
University of Kansas model on the Book 4A test but it 
potentially is far more important. Its special importance 
stems from the characteristics of the outcome measure. 
The Stanford-Binet was developed to tap general intelli- 
gence — a trait that by definition is not sensitive to 
slight changes in environment^ Moreover, in practice, 

Stanford-Binet scores are geneVally difficult, to change 

\ ■ . 

very substantially. Yet here we see an estimated change 
-of almost a standard^ deviation in magnitude effected 
over a seven month preschool program. VJhat accounts for 
this effect? 

Three issues are important. The fir.st two h^ve to do 
with the data and the third has to do with the nature of 
the High/Scope program. First, although the High/Scope 
pyj sites ranked first and second in observed gains there 
was aiiramatic difference between the two sites. In one 
_site the aver age gain wa s roughly 30 points — of some 
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importance is the fact that the four classes in this 
site had gains of almost equal magnitude. The second 
site averaged gains of only twelve points. Although 
both gains are impressive in magnitude the difference 
between them suggests that the effects of the High/Scope 
program may be sensitive to differences among sites in 
such things as pupil composition or location. In this 
instance the site with the thirty point gains is located 
in tfete rural South, has a racial coitposition of roughly 
70% black and 30% white with none of the children having 
previously attended preschool. The other High/Scope site 
is located in a small urban northern city and has a racial 
composition of about three-quarters Mexican-American, 
about one-sixth of whom had previously attended preschool. 

Second, the pre-score mean for the children in the 
southern rural site was the. lowest of the site pre-score 
means. On the one hand, this suggests that a regression 
artifact might account for some of the thirty-point gain. 
Even supposing, however, that the Binet had a reliability 
of only 0.70 (undoubtedly an underestimate for even the 
individual test administration much less classroom 
aggregated means) and asstiming that the "true" population 
pre-test nean was 95 (probably an overestimate since it 
exceeds the overall pre-test mean for the entire PV sample) , then 
the regression effect would account for a little over five points 
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of the thirty point gain.* 

On tne other nanq^ however^ rnere is some independent 
information suggesting that the low pre-test scores are 
valid. For each of the other academic outcome measures 
discussed in this chapter the southern rural High/Scope 
site had either the lowest or second lowest pre-test 
site mean. Since these tests were given by other testers 
than those who gave the Stanford-Binet ^ there is good 
reason to believe that the pre-scores on the Binet are 
roughly accurate. In the other High/Scope site^ the pre- 
score mean for the Binet is close to the overall PV sample 
mean and is consistent with the other outcome measures. 
This suggests there is little chance of a regression effect 
for this site. 



*Kn6wing the observed mean^ reliability and population mean^ 
we can estimate the magnitude of the regression effect. In 
this instance^ We have an observed mean of about 77, we have 
taken a lower bound for a possible reliability (Q.70) and we 
have taken a high estimate of a population mean (95). Our 
approach^ therefore^ will overestimate the regression effect, 
The general formulation is that the regression effect is 
equal to 1.0 reliability times thq difference between the 
population mean and the observed sample mean. In this case 
we have: 



regression effect = (1.0 - 0.70) x (95 - 77) = 5.4 points 
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This argument also deals in part with some observa- 
tions made by SRI personnel at the southern rural site. 
Apparently, the fall Stanford-Binet tester was very 
efficient about his work, spent little time with the 
children and as a consequence , appairently had little, 
rapport with them. The spring tester, however, was loved 
by the children and spent a much longer period of time 
administering the test. The. difference in style might 
account for some of the gain. It is possible th,^t the 
fall tester was obtaining underestimates of the "true" 
scores while the spring tester was obtaining overestimates. . 
The very low pre-test scores for the other outcome measures, 
however, suggest that the fall tester was probably not 
particularly biased. A possible bias on the part of the 
spring tester cannot be so easily dealt with. We note that 
for both the Book 3D and the PSI outcomes, the gain scores 
for this site were either near the largest or the largest. 
This, however, does not account for a thirty point Binet 
gain. Our best guess is that roughly ten of the thirty points 
are probably due to a combination of tester and regr'ession 
effects. There do not seem to be any peculiarities about 
the testers in the other High/Scope site. Thus, we estimate . 
that the true gain for the southern rural site is roughly 
20 points while the "true gain" for the northern urban site 
is rough l: !>L_12—Boiiits-'— — — — — 
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Third, there is some preliminary indication that 
the children in the High/Scope sites are getting certain 
items correct on the Stanford-Bine t post-test that 
children in other programs are not getting right. These 
items have to do with differences and similarities — 
concepts that are an integral part of the High/Scope 
curriculum. An analysis of this issue as well as of 
possible tester bias is included in an appendix to this 
report . . , 

In-^umma ry, we conclude that the High/Scope model 
is particularly effective in producing gains on the Stanford-. 
Binet. ■ We estima te the "true observed gain" to be in the range of 15 
to 20 poi nts while the differences between High/Scope and • 
conventi onal Head Start gains range from 10. to 17 points 
with a " true" effect probably dlbser to the bottom end of 
that ran ge. We reached no firm conclusions indicating 
positive or negative effects for any of the other models 
though there is some indication tHat Bank Street may be 
less eff ective than other modeU and that the University of 
Pittsburgh and the REC models mW be slightly more effective . 
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VIII. Motor Inhibition \ 

. • . . v.- ■ . ■ ■ .■• \ . . ■ ■ 

A. Site to Site Differences ^ \ 

Table VII-17 shows site pre-test means an^ observed 
gains on the Motor Inhibition test.* Pre-test iri^ans for 
the PV sites range from 4.48 to 5.71^ roughly 1 . k individual 
level standard deviations. Comparison site pre-^tesN: 
means have a range of similar size^ from 4.25 to 5.6^^. 
Although the comparison site distribution of means is ^ 
slightly lower bverall than the PV distribution the middle 
^50% of the means of the two groups overlap almost per- 
fectly • The middle range of the PV means goes from 4.79 
to 5.19 while the Comparison site middle range is from 
^.77 to 5.23— roughly 0.9 standard deviations in both 
instances. 

In terms of "gains" the PV site distribution is con- 
siderably tighter than the Comparison site distribution. - 



♦As described in Chaptexf III a child's score on this, test 
was calculated in a somewhat con5)licated way. Fijrst^ in 
order for a test score to be included in the analysis the 
child had to answer correctly two or more out of four ques- 
tions developed to assess whether he \xnderstood the words 
"slow" and ^*fast". The sample used here contains only 
children who met this criteria in both the Fall and Spring 
testings. Additionally^ the tester had to certify both 
» test administrations as valid. The test is comprised of 
three sections: "draw a line", "walk slowly", and "truck 
pull". In each section the child is asked to complete the 
task at normal speed and "slowly". We eliminated the 
"truck pull" task from the analysis for psychometric re a- 
^^ns-, — A— gfe41d^-& score was c a rlc ula L ed 'iyg^aking^^^E^ 
of the sum of the "slow" times (in tenths of a second) 
for the other two tasks. 



TABLE VII- 17 
Motor Inhibition 



PJCd-tciit means and mean '"gains** . (post'-teit inean - prc-test 
mcian) by site for. PV and Comparison groups* Site moans are 
unwoightcd averages of classroom moans. . ' 
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maun 


07*14 


. H« I^as Vegas 


. IX 


4*96 




0*51 




4 






07*14 


W. Las Veqas 


II 




5.15 




0.13 




4 


BuHhell 


08.04 


Portagoylile 


III 


4.76 




6.46 • 


0.59 


% 


2 




oa.oa . 


Moundn, 111* 


II 


4.75 


4.72 


0.73 


0.80 


4 


2 






Ft* Walton u. 


:111 


"4; 57' " 


4. it? 


6.15' 












' Peni^ucola 


III 






0*08 




2 




00^00 


Grcelcjy 


III 


4*90 


4.99 


6.34 


0.49 


4 


3 




05*10 


Seattle 


II 


4*96 


4*86 


0^34 


0*50 


4 


3 


(>orcion 


^ 10*02 




1X1 


5*06 


5.23 


0*45 


0*16 


3 


3 




10.07 


' Chattanooga 


^ III 


4*48 


5*05 


0.70 


0*55 


4 


4 




10.10 


Houston 


II 


4*64 


5*54 


0.44 


0*13 


2 


4 




U*65 " 


Washington 


III 


5*li 


4*i5 


oas 


0*75 


'i " 


3 




11*06 


Pa tor son 


II 


4*09 


4,74 


0*42 


o»a7 


3 


1 




11*08 


«7ohnston Co. 


ixx. 


5*50 


5*22 


0.11 


0*60 


4 


4 


Pittis- 


12*03 


Lock Haven 


IXf*^ 


4*75 




6.2« 




4 




burcjh 


12.03 


Mifflonburg 


III 




4.57 




0*58 




4 


mc 


20.01 


. Kansas City 


III 


5.35 




-6*16 




3 




llhablers 


27*04 


^ Billings 


XX 


4*70 




0*7? 










27*05 


Colorado Sp. 


12 


5.21 




0.62 




4 






27*03 


Bellov;s Falls 


IX 


5.37 




0.50 




1 
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The overall range of PV gains is from -.20 to 0*72| or 
roughly one standard deviation* The Coinparison site 
range of gains is from -••59 to 0*80^ about 1*4 standard * 
deviations. The middle 50% of the PV distribution is 
only slightly more tightly bunched than the middle 50% 
of the Comparison group* The PV range is from 0.15 to 
0.52 points (•? standard deviations) while the Comparison 
site gains range from &.13 to 0.58 points {0.90 standard 
deviations) ♦ 

Relative to the other tests the variations of mean 
site gains for the Motor Inhibition test is larger than 
for the PSI and somewhat smaller than for the Book 4A test. 
Since the degree of variation of gains appears to be re- 
lated to the occurrence of clear "effects" in the data 
this indicates that there may be some effects for the 
Motor Inhibition tost. 

When sites within models are examined two models 
stand out as having a clear pattem of large observed 
gains. The University of Kansas model has the site witJri 
the largest average gains of all the PV sites and a sec- 
ond site with a gain just below the 75th percentile level. 
In the Enabler model two of thfe three sites show gains 
well above the 75th percentile of PV gains and the third 
site is only very slightly below the top 23 percent. On the 
low end of the scale the EDC model has two sites slightly 
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below the 25th percentile. Note also that the only site 
in th«2 SEC model shows a loss of -0*16 gointSi placing 
it at the very low and of the distribution of site g^lins* 
Sites in the ipther inoaiels seam to mha^ little pattern A 
with iftost i!K5dels having both relatively high and low 
scoring sites* 

B, Model to Model Differences -. ' ' " ■ ' ^ 

.The same lour models stand out in Table VII-13.* 
The University of Kansas and the Enabler iRddelg have ^tha 
two largest mean gain scores while BDC and EEC show the 
sinallest "gains". The overall range of gains for tlie PV 
iRodels is roughly 1.4 standard deviations ^ from -So 06 to 
0.64 points . Since only -four children in the RfiC.^iBodel 
received valid scores we;>*ill elimnate this isiodel fro ia 
fatoro discussion of • this*, test . The range without REC 
is from 0.21 to, 0.64, about 801 of an individual standard 
deviation* 

A contrast of the PV model gain tmrniB ^^ith. the ir^ans 
of tiieir Coi^^^parison groyps shovs two statistically sign!* 
ficant dif fa ranees each favoring the Con^parison grot^p^ 
The iifioan gain for the Far West PV groijp is 0^ 35 while -its 



* Be call that the w^anm in Table VII«18 are oalcolated by • 
pooling all children in all of the sites of a Modei-whila 
Ui^ site itaans in Table VII-*17 are ijt^ans of olassroom means 
..Since ther^ ,„art .^.if fere.nt ,.iiuir^tr.s o£. children .,in...differanti . .. 
olassrooiris the two ways of aggregating sooras occasionally 
produce Bozmfnat different results* ThuB, the average class 
room mean gain for the r^C site is -0^16 while the-^ average 
individual gain is --©.©e^ , * ' 
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Cowp&TiBm ' qmup has a mm gain of ®o64 yielding a ^if~ 
f^mnm q£ ^^ZB points statistically significant at thi • ^ . 
•■©*0S l0V0lo . Sin^e only oii# 0f tM two Far W@st sites. 

fer^ntca my well refl^^fc mcoi^trollad sampling bias<.'Iii^deed 

the ?t^afis tttt tte Par Ifest FV sit©, whi^fe "^^as a 6o^arisoE . 

' fTOMp^ Is Qnh£ 0.-14 points hmlm ii^s Conrparison- 

graup naano iiicliEafeiKi is^^td attribute this differ** Z"' - 
' - ■ • ■ ^ # J. . ^ . 

* <^ ■ • ^ ^ji^ 

t© ^fiano^o The i^ea^ii^ modtel shewing -a significant * • 

^ I 

* * 

tte Copparisoii qmup^ Blum all tliree of the ED®. sites'- 

iiaf^ «'-sitfe' Comparisons and 'ill* ^^ach iiistanee tM . • 

BmmmM j^QO^ .raaBon t^-^^iinl, feliat tfei© effect 'my fee ^ valid ^ 

^ - ■ i& ' • 

■ Mhm^ '%%s^ri^od-Escp©dt©d;'\,faiiis..,a» ooi^trastad for 
the artid C©5ifcparii>©r^t f myps tlie F^r Mast mDaf l da#s not 
ste'W' a siffiiifiemt' difference wliile, tte ICC f^del aon-- 

4%fiotlvc-;r contraiifc al^^r^© w>umm •signifioant y©s alts 'In this 

^eolwf-tio ft^CH- child rwi in the Ci^i varsity ©£ Oregon ■ »di^l ^ ,. 

appe^ar tQ gaifn sign^ f iea'ntly: f^orc^ thMi their Comparisons.. • ' ■ 

Insptetefe^r* ut. Tablc^ VH-lf ren^eals that thin differoii€>2 

of 'royghlf 0o30 p©4Siits pa# be »re to Oie poor sha^iitg « 

c^f tiMr Ce.^w^ri^s.on €^i4ldrell thasi^ t© ^ strong ^hcgc^ing for 

tte lisai^o Oregc^P P¥ ctoildreno Each of th^ thrae' 



Oregon Comparison sites gains fall below or at the 25th 

SiJiet^ th# Enablar- group does not have Coiiiparison 
sit^s there is bo way ©t knowing from this tabl© whether 
its of feetivonoes is duo to the model or to tho sanipleE 
of children in the Eftablar sites, The Univ.* of Kansas 
FV P0d©l, which has the largest observed *' gains**, does 

' only slightly better than its CoiBparisojni group in the con- 
trasts in Tat>l0 VII'-IS. It must be noted, however, tliat 

^©nif 11' of the C©^«paris0B children in the Univ* of Kansas 
sitas had valic5 pr# and post Motor Inhibition scores. 

"C* ^'Mjysted Diffarencas Among Grotaps'* 

Tablas ¥1I--1^, 19- and 20 cpntain 9? contrasts* 
Wiirtoen are ^statistically significant* Th© results pre-^- 
BQnt a ver^ m^ed picture. No utodel * stands out as clearly 

i»re -effective than others* The results, howavar, 

' %. 
Bm^m to follow three general patterns » 

1)» Sii-; ^to-dfsls I'Par West, Univ. of hjcizona, Univ. of 

©regon , Ifiqlf^Scop® , miv. of Florida and. EDC) show .genii- 

in^ly mimid results. In bomb iuBtrnx^^B the **ef£ect esti- 

Mtes'' for these w;.©dels ara positlva^ in oth^r* instances 

ncgativci* Only onm of tha 54 astimtes for tfe^se mode. Is 

is statir^tically t^igniCioant, ^ Th# generally small esti- 

iTMateS'^nd th0 PUKOd pattern of results indicate to us- 

that tliore are n^ compelling differences among thaso mod^ls^ 

With regard to rc^e of the si k models this eonclusion . 
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should not be suiTjrising. The gai.i score data in Tables 
VII-17 and VJI-IS indicated that the Univ. of Arizona, 
High Scope and the University of Florida PV programs were 
only of average effectiveness* There were, hov?ever, indi- 
cations that the other models might be somewhat different. 
In particular v;e pointed out that Par West did not seem 
to do quite so well as its Comparison gromp* Our expla*- 
nation for this rested upon potential differences between 
the'PV and Comparison groups* Based upon the data in 
Tcibles VI I -"19 and VII --20 this explanation appears valid • 
A second model (EDO) also did not seem as effective as 
its Comparison group. For EDO we had no ready explana-- 
tion for the difference. And when EDC is contrasted in 
the Multivariate Analysis of Variance with is Comparison 
group (see column 4> Table VIl-19) the PV group still ap-* 
pears somewhat less effective^ though- the difference is 
not statistically significant. Yet when the EDC PV model 

» 

is compared with other PV models^ with the Coit^arison 
classes in general , or with matched Comparison classes 
there do not appear to be any differences. The third 
model (Univ, of Oregon) appeared somev;hat more effective 



than its Comparison cTa^soB in tlTe^gaiit*-isn::ore- analyses ♦ 
llov/everr v;hen the Univ. of Oregon is contrasted with 
other groups its Effects seem to disappear* 

2). Titfo Ptodels (Univ^ of Pittsburgh and PJEC) \seem 
to be systeipatically loss effective than the. other itKsdels 

ERIC ' . H^'i4^ 



TABXiE VXX - 19 



Motor Inhibition 

Model -effect** estimates for the test, Colunuis 1-4 show differences 
between "adjusted*' PV inodel means and sci^e standard. Culunn I shows 
the simple contrasts botv;een the PV model "adjusted** means eind an un- 
weighted grand mean of the model ipeans for an exact least squares one 
way AMCOVA. Coluirj^s 2 and 3 show regression coefficients for each 
modol in an analysis where all of the comparison classes are pooled 
togothor to fonn a comparitson ''Tr.odel". The regression coefficients can 
be Ihoufjht of as rGorosenting the difference between the adjusted" PV 
wodol incans and the* " ad j united" Compari^son **J!}odel" means, Colunun 2 sr.ows , 
the coefficients for a rogrcssiLon analysis not allowing for separate 
slope coc-ef ficients for tl?.e covariatos for the different models. CcIutji ' 
3 shows the coefficients allowing for separate model ^coefficients far 
the VOX pre- test and for percent prior preschool. Column 4 shews the* 
difference between PV and Comparison group "adjtasted** means within 
models for sites wxth both a PV and a Comparison group-. The estimates 
are 1 degree of freedom contrasts in the framev/ork of a one way ANCOVA 
design. Column 5 shows the PV and Comparison n*s for column' 4 analysis. 
A note following the Table lists the covariates used in the anaiysin. 
In all analyses the classroom is the unit of analysis. See te:it 
(Chapters V and VII) for further discussxon of the approac^ies* 





K»tim,ef fccin 
around VV un- 
w<*j<rht<^u w^nn 


Estimated effects 
of PV wdels 
again;;! pooled 
compa r * cUinucn^ 


DF contrast 
PV V. site 

comp* pooled 
hv node Is 


. • 1 

i 

PV Ccrp. 


rnnlyni'? 4 




I.fiboratory 


-0-35 j^^g 


0*07 




-0.18 


4 2 


Ari^sona 




-0.12 




-0.09 


4 4 


Bank St. 




0.30* 




0.40* 


B 6 


U. of 

Oroaon 


-0.02 XI 


*0.14 




0.36 


11 12 


U* of ^ 


-0,07 9 


--0*06 




-0.04' 


7 4 


Uiqh 




-0.^7* 




-0*19 


10 8 


IK Of 


-0.12 9 


"0.07 




-0.17 


9 11 




D,22.-„ 






-0.23 


9 S 


of 

1'} t: 




«0*23 




T 0.-2 5 


4 4 


0.00 












0.24 








Hrand He* an 




5*38 


t 


5.38 





^f&BLE VII-19 

* Statistically significant at the .05 level 
Statistically significant at the ,01 level 
Statistically significant at the .001 level 

1. Only py classrooms are included in this analysis. 
The multivariate F with the-PSI, Book 3D, Book 4A and , 
Motor Inhibition in the analysis is 2.43; significant 
at the .001 level. The overall univariate F for the 
Motor- Inhibition is 2.62, significant at the .001 level. 

2. Both analyses were in the regression framework with ^ 
the pooled Comparison classrooms as .the "dummy variable" 
left out of the regression. Analysis 1 did not contain 
separate slope coefficients for the various models. 
Analysis 2 allowed for separate slope coefficients for 
PSI pre-score and Prior Preschool Experience. Analysis 
1 explained 47.2% of tli|| total variation. 

3. Only sites with both PV and Comparison classrooms 
(on or off-site) were included in this analysis. 

Note; All analyses included the following covariables: 
PSI pre--test mean. Book 3D pre-test mean, Book 4A pre- 
test mean, mean age, percent black, percent Jlexican- 
American, percent female, mean income, mean household 
size, teacher experience in Head Start, teacher certifi- 
cation, mean mother* s education, percent prior preschool, 
average staff working conditions, whether the site is 
El or Ek. In the analyses in column 1 the variable 
**site administered by CAP or by Public School", was 
also included. In analyses of the Stanford-Binet , the 
Stanford-Binet pre- test v;as also included as a« covariate- 
these analyses used only Level III sites. In analyses 
of the l4otor Inhibition only classrooms with valid Motor 
Inhibition scores for both fall and spring were included. 



TABLE VII- 20A 



Selected Statistics for Matched Classroom Analysis of the 
Motor Inhibition for the 5 Factor Match 



(See Chapter V for description of matching procedures.) 
Column 1 shows the number of matched pairs of classrooms for 
the model. Column 2 shows the covariate means for each model 
il*V prc-test - Matched Comparison pre-test) * Columji 3 shows 
the unadjufjtcd dopendont variable means for, each model (PV 
patil-totit - Matched Comp£trioon post-toiit) . Columns 4, 5 and • 
C rihov; adjurjlod dependent variables for each model (the DV 
adjujitod for the covariute)^ under three conditions of. eiiti- 
taalcii of the reliability of the covariate (column 3. estimates 
Uie rel i*4biJii.y ar; 3,00, column 4* iifi 0.80 and col)5mn 5 as 
0.00), The Lord'J*ort<»r correction ir» used to "correct" the 
coviiriate lor xtu xeixability. 



I Covarxate 
t Mean 
jPV Pre-Te^:t 
[- Comn, Pre^ 
Test 



Unadjusted, "Adjusted Differences" 

Difference ? (PV Post-Test - Corp. ?ost-T£S^) 
PV Post-Te5n_^ (Adjusted^ for Pre^J^ef t^^Covari :^.ncf* 
- Comp. j Covariate^ ^ Covariate " Ccvar:.titc 
Post-Test. |Rel. = 1.00?Kel. 0,8a Rel. = 0.6: 



Far j ^ 
Laboratory 


0.04 ^ 


• 0.03 


0.01 


0.01 


-0.00 


Arizona j B 


—0.22 


-0.04 


0.07 


0»iO 


0.15 


Bank St. j 8 


0.08 


0.33 


0.29. 


0.28 


0,26 


Vuiv. of jj^^ 
Oreqon * 


--0.27 


-0.12 


0.02 


0.05 


0.11 


Univ, " 7 

H'i qh ' " 


-0.46 


O.iX 


0.34 


0.40* 


0.50** 


-0.04 


-O.OIJ 


-0.08 


-0.07 


-0.06 


-0. 3B 


-0.18 


0.02 


0.07 


0. 15 


j:b(: ' ^ ]i 
UJiiv. ^ji™™" 




-0.01 


0.00 


0/00' 


0.01 


■ 0.20 


-0,28 


-0.39 


-0.42 


-0*46 


ni:c j ^ 




0.03 


-0.27 


-0.34 


-0.47 


r;nablerr. ^ 


0,11 

^ ., , „, ; 


0,31 


0.25 


0,24 


0.21 



* faatir.tin;il3y si^inif acant at the .OS level 
** raaUf.ijcal Jy t.ifini/icant at the .01 level 
*' *5;tatii:*tjc^iUy nxtjnif icant at the .001 level 

^'yh(' overall correlation between PV x^re- and Comoarison prc-l|est 
milfhed c3rif.sroD,^ meajiures « ^^j2. 'The overall' F for the tc:.t of 
homtHprnoiiy of the covariate rcgrej;sion coefficient ^ 2.04 * 

HUi' rttjrf^r.uion coafUcxunt for thu covariate for the analvcis with 
ti'}hihn^iy ir, ,j cMiiMted az 1.00 '0,^i2; v/ith r^* ej^tir^ilod ar; 
O.Uh the co<'f3 ic^ejit n O.Ob; Cor r^ n bToO, the coeificient. " q^,. 



ERLC 



TABLE VII-20B 

Selected Statistics for Matched Classroom Analysis of the 
♦ Motor Inhibition for the 4 Factor Match 

■ ■ ^. 

(See Chapter V for description of matching proceduries. ) 
Column 1 shows the huniber of matched pairs of classrooms for 
the model. Column 2 chows the covariate means for each model 
(PV pre-test - Matched Comparison pre-test) . Column 3 tSiOws 
Ihci unadjusted dependent variable meann. fot iiach model (J?V 
por;t-tost - Matched Comparison post-tost)* qj^luitthd- ^H.S^/ihd 
C s;how adjusted dependent variables for each model . (thejDV 
adju!;icd for the covariate) under three conditions of entt- 
mates of the reliability of the covariate (column 3 ea^lmateg 
the reliability as 1.00/ column 4 as 0*80 and column S^n 
O.OOJ i The jLordrPorter correction is used to "correct 'J the 
covariate for its reliability. ' . 







Covariate 
Mean 
PV Pre-Tost 
- Comp. Pre- 
Test 


Unadjusted 
Difference 
PV Post-Test 
^ Co'mp. 
Post"-Test 


\FV POSt-TC 

^Adju8tcd_4 
Covariate 
Hel. » 1.00 


]8ted Differ 
JSC coropp 
^or Pre-Test 
Covariate 
Bel. » 0.60 


ances" 

rOB w— i vto k> / 

jCov^arjlanco, 
Covariate 
Rel. « 0,6C 


Far West i ^ 
Lcjboratory 


0.15 


0.41 


0.34 


0.32 


0.29 


Arizona | 7 


0.04 


-0.01 


-0;03 


-0.03 


-0.04 


Bank St. ; 8 


0.17 


0.25 • 


0*17 


0.16 


0.12 


Univ» of gQ 
Orccjon 


^0.20 


-0.24 


-0.14 


-0.12 


-0.08 


Univ. of : ^ 


«o;25 


0*24 


0.36* 


0.3S* 


0,44* 


Uiii vr"o/ 
Florid.i 


9 


0,00 


O.A$ 


0*16 


0.16 


0.16 


,9 


-0.26 


-0.07 


0.06 


0.09 


0,14 




oai 


-0.00 


-0.13 


-0.14 


-0.17 


Univ, of , 3 
PittJ^Uurgh 


-0.11 


-0.5.5 


-0.50* 


-0.49 


-0.47 


m:c \ ^ 


0.45 


-0.2X 


-0.42 


-0,47 / 


-0.56* 


linablors ^ 


0.39 


0.54 


0.35* 


0^30 


0.23 



♦ statistically significant at the *05 level 
♦* Statistically sicjnificant at the .01 level 
♦^^Statistically sicjnificant at tao .001 level 



'^Thc overall correlation between PV pre- and Comparison pre-test 
matched cliif-sroom measures ^ 0^3* overall F for the test of 

homocjonaity of the covariate regression coefficient « O.Sjj i. * 

-^Thc ri*ijTQ:u/ion coaffieiont for the covariate for the analysis with 
r(ilu\)nlit / (r^il oiitiuuitQd ].00 - 0^7? with r^.^ e»timated ao 
0.00 Ui<- cofi f icic-nt ^ o*%si?for r^j;^*^ 6.(>1), the coefficient * Q//^ , 



REC will not be considered since the sanple size is so 
small. Thei University of Pittsburgh model, like REC, has 
only one site so we cannot make strong claims about its 
effects. Yet for all but one of the contrasts in Tables 
VII-18 through VII-20 the estimated effect for this model 
is negative. Of the negative estimates the range is 
from -.23 to -.50 or from one-half to one individual 
level standard deviation. Due to the relatively small 
number of children (18) and the small nuirber of classes 
(4) only one of the effects is statistically significant. 
Our conclusion is to suspend judgement about the effec-5 
tiveness of the Pittsburgh model for this outcome measure. 

3). Three models (Bank Street, University of Kansas and the 
Enablers) appear to be of above average effectiveness in teaching 
motor control. Although Sank Street appeared only to be 
as equ.ally.^ffective as its Comparison group in Tables 
VII-l? and VII-18 it has a consistently positive pattern 
of effects in the contrasts in the other tables. All 
three of the contrasts in Table VII-19 are statistically 
significant indicating that the Bank St. PV classes 
generally have children exhibiting greater motor control 
than the other PV model classes, than the Comparison 
classes in general and than its Comparison classes located 
in the same sites (the elimination of one of the PV sites 
because it lacked a Comparison group of classes accounts 
for the difference between the effects in column 4 of Table' 
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VII- 19 and the effects in Table VII-18). In the matched 
classroom analyses none of the effects for the BapSc" St. 
PV g'rolip are statistically significant although they are 
all positive. The overall range of effects for^Tables 
VII-19 and vri-20 : are from 0 . 12 to 0 . 47. The University 
of Kansas also seeips to show a generally positive set 
of effect estimates, \(e noted earlier that both of the 
Kansas py sites had relatively large observed gains* We 
also noted that the Kansas Comparison saxrqDle of children 
was particularly small. This suggests that we should 
disregard the contrasts in T^les VII- 17 and VII--18 and 
in column 4 of Tabl^ VII-19. if we do this we find posi- 
tive contrasts for six of the eight other instances with 
significant results in five of the six positive cases « All 
six of the positive contrasts are for the matched classroom 

analyses where the range of effect estimates is from 
0*34 to 0.50 points— from two-thirds to a full standard 
deviation. When, however, the Kansas PV classrooms are 
contrasted directly with the other PV model classrooms 
or with the Comparison ''classes overall the estimated 
effect for the model is essentially zero. This contra-- 
diction in results may stem from the very low pre-score 
means for thd Kansas PV sites • 

As we noted earlier the Enabler sites all seem to proviqe 
greater overall gains than average on the Motor Inhibition. 
Moreover, when the Enabler model is contrasted with the other 
models its effect is the largest* Finally in the matched 



analyses the effect for the Enabler model is always positive 
and while significant in only one of the contrasts never has 
an effect of less than 0.40 standard deviatipns. 

We tend to be optimistic about positive effects for 
both the Bank Street and University of Kansas PV models though 
we cannot reach a firm conclusion. Our optimism stems in part 
from the r^^ults presented here an<l in part from the fact 
that it makes sense for both of these models to have an effect 
on a child's motor control. Although a Bank Street classroom 
is not structured in the same sense as a University of Kansas 
classroom (with academic drill) it generally has a quite for- ' 
realized set of conventions regarding the nature of adult-child 
and child-child interactions. Children are taught to have 
respect for others and to be self-conscious about their 
aggressive bfehavior. Such instruction should bear a relation 
to motor control and the: inhibition of impulsive behavior, ^he 
Motor Inhibition test should tap this dimension. Similarly, 
the reinforcement principles effected by the IMiversity of 
Kansas model might tend to encourage children to increase their 
motor control. We have no explanation for the apparent success 
of the Enabler model on the Motor Inhibition. 

In summary i there- do not seem to be any models which are 
definitively more or less e ffective in aiding in the devol- 
opment of nofior control. - There is soiae^^indicatioR, however » , 
that the Oniver sity of Pi ttsbu rgh_ model may be relatively less, 
effective^ artd that Bank Streef, the University of Kansas and the 
Enabler Riadi'^s may bo yelativ>3ly imro ef feet i ye than the ether 



^ models. , ^ 
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IX. Summary of the Effectiveness of Different Planned 
Variation Moaels 

Table VI 1-21 crudely summarizes our findings re- 
garding differential isodel effectiveness. The eleven PV 
models are the rows of the table while the five outcome 
measures are each represented by a column of the table. 
The cell entries indicate effectiveness relative to the 
other PV models and to appropriate' conventional Head 
Start classrooms. Pour categories are used to indicate 
whether- the model is : a) . Probably less effective than 
average; b) . Of average effectiveness! c) , Probably more 
effective than average; and d) . Mroost certainly more 
effective than average. Six general conclusiohs may be 
reached after inspection of .this t^le, 

* We began this chapter 'witiv a" major esmectations 
that ther© will be few strong differences among th© models 
in effectiveness as assessed by our five outcome measures. 
By and large fchis expectation i^as realized. Table VXI-21 
clearly indicatas that for eacfo of the ©utcou® measures 
we have classified the majority, of tlje .models as having 
average ef feetivt^ness. Moreover* no model stands out as 
either tmm or loss effective than the others on mre 
than twd'\of the five outcosss* In the crudest tem^ 

there ark^RQ overall '^vlnners or losers. 

I 
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TmhE VII-21 . 

Suironary of Planed Variation Model Effectiveness on rive 

Outcome Measures 

Zero (0) indicates luodel is of average effectiveness on 
outcone measure. 

Minus {-) indicates iRodel ma^ be of below average effec- 
tiveness. 

Plus {+) indicates model ma£ be of above average effec- 
tiveness. ' 

Dottle plQS C+-i") indicates model is probably highly 
effective. 
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2), A. second mom tentative expeetation sagq©st@dl 
early in the cha pter was that which ei?^hasiz©d 

- ' academe driXl CQg^ined ^itfe systematic j^inforeeiteist 
tuiQuld be TOO re eff estiva than" other Models on th© fpyg ^ 
cognitive onteonte ineastares. This e'KPeGtatioii^ was r@al^ 
ized only for one ©£ th @ ionic so gnitiw laeasures, Onlj 
for the Boolt4A measure—a test assessing knowledfe of 
letters, nuuerals, md pliape naiafes— is tli©re evidence 
of greater effectiveness for th@ models eji^hasizing 

drill and reififorceKetit. the- University of Kansas siadal 

is the clearest en&mplQ ©f t^is finding. We found it - 
•to be clearly s«p#-rior t© all of the other TOdels and ' 
t© th® CofBparison classes in its ©f f©ctlveness in rai§4nf ' 
Book4A test scores „ Tne otiier smdelm we rated as 
ep^asiting. academe ..drill CIMi varsity of Oregen anil tin! « 
versity of Pittsburgh! both appear to be ab©v© average 
in their ir^act on this test." No ©ther fr!.odel has ari 
afeovo av»'irafe effcet for this test. 

On th€' ©£her eomitive tests there is^' n© iadicatim ©f • 
special effectiveness ©f these thm& models <, Only the Uni"- 
versity of Pittsbm-qh mMel on zhQ Stanford Binet showg .an 
other than <a%»erage £ect» These findings m@ at soim 
variance with the findings ©f ©ther researeh©rs in th© p,r©- 
school area Csee . Bissell, and i>rnit©, et al, 1S?2K " 

These researchers indicated that ther© pay be ,a general 
positive effect of structured academe e^basis and 

ERJC '-'^ 

■^1 A 



drill on eognitiv© mmtm.^ Our data, li©we¥#s> itiM&wt& 
that the,©ffesfc is specific ratiieif than ^n&ml^ In 
patti'ealar it appears as if this app^saefe say fee m-ai^e ef^* 
feetive for iisgarfciaf inforftataon that is easily taii#fe 

through sfstaisatie drill whii© it is ©rsly ©f aw&sfe 
effeefcivs^ess ©fciicir- eeffiifciv© aieteaso Of the f®ar 
09fiiiti¥© tests £l%® 8©0fe4A -test Piasfc elearif a;is@gses 
speeific skills, fi"?© ether tests # .parfcieiilaiflf the ?SI 
the St.aafo«l Binet* assess genera I iiif©r«at4©ii. and 

3^ „ One aj-oiisl el©ari^ stands, ©afc as, gage ef feefci^a 

thm th{s,efchers in Staafog^ Bifiet t©gt seog^Sp 

The, tfigh-- Scop#, P¥, gjsael, appears t@ in&mmm Stante^^ 

• ' ' ' ^ 

average effeefe ©£ othmw P¥ aad Coii|^aris®ft 'psiels is ©m 
. th#. @rd@r of tw© t@ "points ©r ^©aflily 9«2 standard 

dematiosis^ I'll© ©ffect ©f the Hifli Seep® wmAel is par- 
ti€«l(arlf sferonf in mm Soutlieris. roral sit© where the 
{isgastirea average gain is slifhtlf over tbljfty. points o_ 
Although we em probably attribute ^08»e of the aisasured 
faiii! t© tester afiil r^jfre-ssim effeet© the ®e©rr#«£@<i'"* 
ftiirt is stAU.oa the ©rder op a vesy substaatial twenty 
points. Pselifdnary analyses of the item profiles of^..^--' 
chiiasen in the Bi.^ft Scope sites indieates m^^^® fains 
' may partly fee attr^l^uted t® the egsph^^rC of . the Hifh 



SmpQ wm^Ql ©n tii© ^Qncuptm &i ®iir41'aiel.tiefi and differ- 
©E^#.^o CS^^ Butlmi^i in pr^par^atlqfi' m a separately bound ^ 

- ' file partienlar miiQ^tivmrnB of th© Migli B^Qpm i^^dal- 
« the* Staiiford Bin^t d^oc^e not appear fco gan^ie^alissa 

fomr mmmnmu .mmtm tte ir^i^i apf^am teo fee of ©^ly 

m w^m^ ln^ieatiop tliaiK tlie El^ti Soope itt^del my fee ©f 

hmqh^ md-'BEC} atwrnfe f@r 40%. of til© IS mllm i^s'.^abte 

f'ifctfjfet:?^^^ appears aB@v# ai^mft m th© Book4A a^d. Stap-- 
fori ©ii^efe t^s& and b©l0i^i mm^^ om th^ Motor Inlilbi*^ 
fei^n REC app€?arii bc-l©i^ av©raf0 oii tlie Book 3© and 

Sq0?c4a t€"Sts abo^^e ar^rag^^ mi th^ Stanford Sinc^fco 

€^tter i.s rated, as ©tter than average 

tftars tM0 @l tfc P^af?iires6 fferc^# things ate ^omn^ii to 
i£C apfi Pit,tsby.rfftc Eaefe mmw^ £@w prograraped 

iimtnirtii^^r^* vf^s a first f@ar li^d©! in If a^d 

©acti Um milf mm t&itm m this study ^ Mfcteogti tfm first 

t@ ¥ic^ %im th^M ^aeli i^d©!. lias ©ftif ofie site as tlie 



•priaeipal risason ttiat these iHsdels have more than their 
share-^ of "otJ^eir thak -average" effects. .. As we note through- 
out th# ehapter it_is eoirap,oii for jfisdals with two or more 
sites t® mhmt considerable site to sit© variation in 
effects o Tlsi& way hm due t© differential ©ffectivaness 
©f tfe© »dels iin aifferent sites or to meontroliad ■ ' 
biases in ©ar data» Wiiatever the reassyj ©yr .inclination 
i© t© b©- vesy sJ(i©ptical about attributing clear effects 
t© any model ^^itlj ©aly. -oirae sit©. 

-SK Mi iiDiels ar© ratitd as showing average' effec-? 
tiwness tli© PSI testo m had i^ot escpeeted this 

^ result sin^e our preliidnaiy analyses of the PSI indicated 
that it is probably ©«r mst reli^l© tmmmm, in retro- 
sp©et, h©we¥er, is?© smspeet that th© reason for the lack 
of clear differenoes among isodels ©n"-the PSI is due to 
tlie nature ef the 'test itself The PSI was d©vel©]^©d, as 
a f#«©ral test to assess the overall iwpaet of ■preschools -• 

OR ehildrea* ■■■■'As. spch it attempts to mmme- a wide rang© 

♦ - 

M mMllB probably rendering it r#latiwly inmmnBxtivQ 
' fe© parfcicular differences among cyrricyla^ Taii^s xt is 
prtsbablj* mr© appropriate t© the tasks of assessing the 
Qmmll ayera^e i^paet c?f preschosls (see Chapter I¥| 
and of individual di f ferer^ces ameng children Csee "C©fni=- 
tiw Ef£€?ei3 of Presciioel Medcrls ©n Bi^f©r©nt ivpes of ' 
Childr©n"K 
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6). Threa models (Bank street, the University of Kansas 
and the En abler models) appear to be above avexfg^ in effective- 
ness as assessed by the Motor Inhibition test* |wa argiie in 
section VIIZ Of this chapter, that there are substantive reasons 

for the result relating to tij^a curricula of Ban| Street and 
the University of Kansas* W# do not know why t!|a Enltbler model' 
appeared more effective thanjmost other jiK^dels. \ 



Chapter VIII 



MAJOR CONCLUSIONS 



This dhapter b'ti^tly ^smm^zizeB tmjor conclusions oi 
the report* axtansive suim^r^ of this report and tha 
other three preiimnary reports on Head Start Plmfted 
Variation^ 1970-71 is being prepared by th0*Hyro?i Institut^^* ♦ 

Thrae i^ain^ questions ware addressed in this reports 

1* I'&at ara the short term effects of a Head Start 
•experianc^ on children? 

2» Ara tJhi^^re discan^able differences foetwaan the 

effects on oliildxejn of a Head Start Pla^^ncid Vari'a- 
tiona a^^erience , and a conventional Head Start 
astp^ri^rioe? ^ 

3^ Bo Plarmad Variatiorii ridels differ in their effQCtn 
* on Head Start Ciiildren? 

Five jaaasnred o^Jteoires ^^era ?if:>ed to ammmn eaeh quastion^ 
The PSI* XB B ^^jTmral stamiardi.m4d a^iemmBUt test for )pra^- 
school children. Tk% UW Book 30 a^r^^Yll mm te^ts 

of 2!^prfCifi€ a€hi€:v4\^i^t areas ^ Wie Star^ford-Biiiet is a ^ ^ 
well know^ t€et o£ gcm^rai ^ intuit i^n^e'' ^ Tim Motor Inhibi-^ 



other tfirce ropo^rti; an t.iiis mriQ^ ar^ eoncc^mc^i t^ith tl^e 
quality of tli^ aat^^* the ^s^U'-^ of ir^ple^^ntafcion and intarac^ticms 
mtm^mi i^^ograr, ana cfeild €raaracec:ri'St^€!S ^feich affect cogi^j^tic^i^ 
©yt€oi?^i^s " . 
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tion test 'assesses a child's ability to control motor 
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Wifcfe -regard to tii^ ^mmtxon of short tami af fects of 
Head Start wa reach four coneXusion^* (Sea Chapter IV for 
details) 

1« The Head^;Start ,;f xggjte iiieraased 
gfeiMr#ii*s t^st scores, or^ ali fiye.ou^ 
0B. foisr oi thm fiv0 outcop^ ^asiires ohildran's 
scores w©ra astirrated to incraana ^naturally** over 
the Bmmn or eight i^^ooths of tiba Head Start pro** 
gram, fiiue^ even had tlie ciiil-dx^n not beau e^jpoi^ed 
to fieadi Start # their ecorea would have ri^ei^. For 
tw of these tmmmmw CPS I and Book 3D| the Head 
Start ^xp^x:iQnm was estimated to doi^Ia tMe "^'mzmaV^ 
rate of. growth*. For two 0tliex...jGsaasMreii. .CBook 4 A and 
ttie Motor iTihitoition^ teste) the Ifcad Start e^iperience 
%?as estimated to better thm triple tha ""natural'' 
rat«? of growth, imimmmtB attribatatoie t€> Heat! 
Start ranqed fmm 0»26 atai^darcl dc*«/iatic»iiia- (for tts^ 
Mentor Ifiriibiticsri test) m standard dft-vlati©iss 

ifsr th© So-ok 41^ tesii, CM the fifth TOa.s«r€, the 

I 

Stanforcl-iin^t r ©or <?!^tii^at€?s indicate that the 
■ Dcorao of miMmn in thi% sair^pli? ^oyld haw "'natural ly'' 
d^ereasi^d t>y abc^ut 6 ^"26 standard #?¥lat4c??iO had ttmf 
m)t att€'r4ded H^ad Start * f j^f ?!^^£^£BH: 
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Head Stanfor^-'Biriet saoras 

rough ly 0 , 40 s t anda rd vi at Ions ^ 

Children *who had a prior preschool e3qperienc% 
gained lass overall V'nutur^V^ »f Head ®isa^ 
raiati^d gro^^th) than children for whom 1970-71 
Head Start was thair first yaar^of prasohool# This 
effect held for all outcome ^ Hieasoraa- and for mo^t 
of the subgroups ^twJied in Chaptier IV*- If, howavor^ 
.wa allocate the total qain^ for the two groups of 
childrafi between **natural groi^th*" and the ftead Start 
e^p^rumQQ^ we fiod that the effects- attribytal^la 
to Itead Start am romfiily equBl for childran with 
and v/lthoutfc prior preschool experiences* This, indicates 
thafc tte expected "natural growth" for ohildrtjn 
with prior presghool eKpe fierce io tess thaw for 
children witJ-iOsit- prior presehool« Th«» prior 
presschoGl esepericmce appearecl to reduce differ- 
cne'ofi 1ft t€?Gt scoroc to#tw<?en children of different 
ages. In other %^orcis, a cowrc^n preschool 6*xper- 
icmce partially metcmmQ the effect ©f age 
MifemnmiS apomg chiielreri on the fivo o«fecorr« 
^alHir^-^fe- Sfrfi?'^*^' Riipport fer this notion cof^a from 
the* fatt that -hsinmcfifi on Four ©f fche fiva ojjtcona:.' 
Measures are' ibrn'^hat sffaikrr at post-fce-'St tiiise than 
at pjc*-?" ten-it fti«!<, ThaG imM^attm th^il fJi f R-rf-riCes 



among children are Xqsb at the end of the preschool 
program th^n they are at the beginning of the pafogram* 
Preschools may^ hava a '*fan**cloBe" rather than a 
"fan-spr0ad" offoct oh children* 

Children ^ho would enfc#r firot grade (Bl) dir^jctly 
from Head Start tend to gain more than children who 
would enter kindergarten (Ek) .dirc^otly froitt Hoad 
Start on tho Book Book 30 1 PSI and fi^tanford^ 
Bimt tmts* On the Motor Inhibition t^nt tho Ek 
children gainc^d mom* (The average age o£ El 
children v/hcm they entered Itoad Start was 65 inonthn 
— Kk children wc^ro roughly oni^ y^ar yowiger*) The 
greater gain for Bl cbildrcm was mo&t pronounced ' 
for the Book 4 A test and laa^t for th^. Stanford- 
BiUQt* it^m% the gaino attributable to Head Start 
ware eKan^intni^ the effeet appc^aro to strengthens 
tiiough they arc* still mnall for the Sfcanford^-Binatf 
Thooe Qti€^ats are probably dwi to "a eoirtoination of 
fcv;o thin<^|0* Tiroti the? - lurqor galriii attributable 
to Hc^ad Start for El children on ttm aognitiv© 
tmmumn and partlcalarly t\m Book 4A taot {a 'immirm ' 
of ic^tter?^^ nuTOrals^ and ribape nmnm^} maf bt^ dm^ 
•to older childrc^n^a advanc^id academi-e riiadinose* 
Sf^ccHMl, thc^fc? n^ay bQ^a grcNutPr inl;^rc>sl by ttead. .. Start • ■ 
teach#rs in Kl sitoQ in preparing ^hildmn far r<>ading_,. 
and arilhK^tie* # Mi ^i|lt , 



4. .There seem to be no ccmsisfcent (differences among 
'Mexican toeriaari, black and white chiiaren in 
• - their Head Start gains on th© five o:uteo«,.,ii»aiaiiros.-. . 

In Chapter %V we disauss the ntethodologieal proe©diir©s 
used to arrive at these conelwsidns. Sine© we did not have 
a group of "control" children (chiXdrc»n who. did not Imm the 
benefit of an Head Start oxperienccs) our egtiiwition procedwres 
relied on natural variaticsna in proseores tow children of 
different ages. The reader, ther^for<a, is warned to treat 
then© data as roucih estimates and to. evaluate . for himscilf th© 
asswroptions of the procedures. 

The second mjor quest im regards overall difftjrenees 
•in effects for. Planned Variation and 'conventional Head Start 
pragrarrr,. This qumklm is addressed in Chapter VI „ Afc 
the begfinnittf of that chapter w© atgw$. that th© question has 
very little import anc«?. For while* w© »ic|ht esspact ther# t«? 
bo- differenees among FV prograws in their ©ffc?«sts on the Civ© 
CHiteonM? iKensurt»fi, we have little mmmn t© swspcict that there 
should toe? syr.tewatie MiMtGUG^Qm between an overall PV Jffeet 
and an gwral^l c;>ffoft of eonvontional Head Start prcHjraPSa 
Thi@ qucfstion, like? mmk total pmqram impml qm%tiom» 
i ot a U f obs euros nfB Ummt i e di f fts rences mum t r«-att!Mj»l I; 9 „ 
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The sole rationale for studyifig the gtifestioii was to deter- - 
mine wheth<sj? the extra fmids aXloeated to PV Head Start 
prograros had a eorisistent ©ffcict on the ineasiired outeomas. 
Ottr conclusion supports the findings of a large nuii^er of 
r«»cent research ©t" forts whieh have failed to detect ^ any 
systoraatie relationship of gross expenditures to variations 

in oute©P«s«. ^Jg_ff^l£ijj£^fe!}gff_ arc? no differeneos in ef feefcs 

between the P¥ programs (taken together) and/tho, CofnjparisQII 
Hoad start p£egr£wj»n^ 

The fcliird question addresses difference® among' PV pre« 
grams in th«ir effects on Head Start children « .We reach four 
iwalor conelwsions m this area, (Bm Chapter Vtt for details) « 
l« There a»e a relatively spall nnjiteer of difference's? 
in of feet© auieng prograiK that are ©£ snifieient 
stabilitj^ md mizQ for m to m-^Qct a null hypothesis 

We rf^TOgnia^e that tter^ iiiay hot mmv^ mmm "^txm^ 

iliftQ-mnimn mm^ ttm mMlels cm the ii..? oiitcomt 
■ than ri?p©rt^ al^a roeoigui^^' ihat 

The tim 4ifimmn&7^fJr' we foimd an> seatt^red 
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Ope tentative expect at 1©?^ in o^r aitaXysi® was fcliat 

mg^e^s. g!!^J^!lg!e„.gi„fcj!g gQ^"^^^^^ tests ^ CPSI^ 

BmM m and- th& Staii f ora-Bingtl^ this expegtaMm 

^i^MSMi^l^^J^S^l^^ tliifee »dels wliicli 

fit tlus ex'iteri©ii,, ©ne CUftiwrsity ©f KansasI stands 
©wfc as beiRf imm effcetivc tlian all ©tlier tm&QlB 
.m imp^tting kna^leage @f letters, ii«8?©rals, and , 
=shaf€? nmmm as nieasyrecl by Besfe 4^8 effeefc 
©f tlse iasisas itgi©! «as )ip tte ©rder of ©^fS t© 
stattiidra aeviatieas^ the tw© ©tJwr Models all?© 
fitliaf the cjrstericm ClMi¥©rs4tf of ©rcf@ii df?a 
|5n.i\*ers4tf ©f Pifetslnir#sl elemlf above t-lie 

ave rages «?£ the mtlnn nm^Qls in effect a wne&s. 
the 4 A ©titC'g'EW" wjastire, 

iliiao ©tJw^r P^^deis iji W&^^^^^^^__^^^~_ ^^^^ ,f ^.ffil^K^. 

mn&t^. «^ C'stapatf^ that "'trsie'" siai^P f©2r cliildreti 
tiM* Higli Se©pe P^clel a^erafeil r©ii#lf 12 t© IS ■ 
^mnts i^kU "fcrese'^ ga^«s f@t- the ©tiser poctels 

att€?is|?fcf; reasons £@r tiM? s««sess ©f tlic^ 
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High Se©pe m^ei ©n tit© StiSHf^rd^Bisieta 

4« €^tii©r in mc data i*erc less Srapatie 

ihm tMs ^mimmitv^ &S Kaasos* mssSel effect on Bcals 

4,A anil liifli Scofc rtsael*s effeet or fcli© Stafifori" 
B-ifiet«. C:a ©tie ©istestt© lEe-aimr© Cthe PSlI , fmnd 

r@r til© &the-r &utmms tm&'smes we i&md iridieati©as 
that tTO tlisre© miclel© ste^'ed fitter ab©m* @r 
b<s^l<Qi4' at'erage ef feet4*/eiiess» in pest-^ anst access 
■ titie ''effeefcs* differed fxen tlm avara^© leae 

sense. Tile ^eiimts" afsi?ar b& related to tli© 

er£K£||u tne laefe e>s differential "uffegts" f©r tMe 
pgl ifiiieate-^^ tliat taffits ilesigneii fe© assets the 

iRS€*Re:^s.s.i%-? mriMsfieas ia earrigisla. ft; aepe^aes 
tfeat ue^id rrs^fCf hirthlv spe^^ifi-c ©et^ecssma msmmt^^A 
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Issfeifi^ Be MafceMpg^. t©^ rfeg^m^e bias, in ofcgeryatieiial , stisdiegg 

seh&^l ffeanso lis FiasiafaR* ©t alo Sfadies of feh ® 

Sta^f©fd Beseareh Isrstitnteo jyaloatieB ©£ .tlie Hafcieiial 
- '^©llw^pttmmh Pte&tmn Itl ^--fl o". fieSlQ FmKg Calif ©ri^iai 

Stanford Research iKstitAJte^, iwflfw^ntatmn&i Planaed . 

j SferarRSi, Sp Pegogt ©ii gres-gheel pge-fffapsg fhe_egf©et ©f 
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. Seaigg £er thm Thisd mviBmn Fom'^-k ^ ^o&feeng- * • 

HOttfist©fl«Mif £.|j17^66'<> ' ' ^ 

Th© Stija^ . ..®f_ffg|!j^Q3L ¥agiatei .©P,s in Head Sta rt l§69o 

©ffeefe c@rif©lati©ii» Glass (m^l Proeeedings ©f tlie 
. Waited States Civil Mfhts C&mimim^ Racial IsQlafcien 
Kepnrt,pf^ m& Pe ggy . PrQSgheex- Fre^C'cto mn mm^g 

lemmm^ in pmmch&Ql .Q^mmtiQUo tm. 3^ Staittey' 

- ' ' ' f 
W«iit©# Sa# ©t alo Federal ggQsraF ig for Yeimf Cteildrens 
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as pr^> mn4 p^st-tet^r^So - fhm wmmum% are i^irleflf described 

m&d ^mmd \*^£i^tim. Oata*** In order feo fee inelitded 

in an amal.f:s4S// a ^las^map^ Car a child deg^ndiPf « tSie 
©f aiial-^^'^'^^sl mmt^ im%m had i& %?aiid ptti post-test 
sfbrcr slic p^rticiilaf ©iiteoiie is^asy^^<. ¥aliditf warn 



tm3 Ja — an adiiewpent tost ass#@siiif 

ep'D:fe_4^ afii a€3i^-€;vefms:fit tC:S.t assessing 
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5« MQtor IwAiibitim a i»sasiire -assessing -a ehiM*s 

ability t© mnts&l his' motor behavior, 

All tmsi^mms, ©f child sfc^Eaceeristics are iatetsa fr©P 
the Class r©0ss Iftf©r7?jati©n iomi (mm Chapter I IK Ctsild 
ctMractesisties w&i'© caleuIatecS by ©ri tlie -:;ricii¥id5i63l. level 
and ofi tfce elas-sroor?; agf^egai;© l-©v©l^ The deseriptien ^■ 

iw^ir~fi; "TtJ r"Tn r Tfiiiivra all ' ^ 

a re>i?i of tt%ti ci-?dre4Cteriscie5 ?©r tticj cMMwen in the class" 
resrs fe-as eeisi^tited^ In jiasteances where the characteristic 
ii'ai a binary varaai^le leof. sexK tn© elassr©op tmm cm 
als© be- 'tristsgl'at ©f a« a prQpegtions @g peggfin^ageo 

3'^ gjgo ciess^-rally t^»j© d«i?yEy variables were ysed 

i,r<«aic.5ti«g f-«iiether the ctftili wif,;? Bidcli or net, and 

*JSMi?Xlf,=JdiStsJ2l ^' van-aisle assesfs&yif the nttsifeer 
©f fears csf sehesHfie ^ siJind's f?:9t!ier has c©?f?pletede 
Tli# rfijRiSfc froia 6»2© iKdieatiof nmis&r ©f years ©f 
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fr#at©r than $f9S0.O'e. 
r airily Bim& -» h 'mriafeie indicatiisf tfce aiMjer ©f 

wtethejf or not a cliiici had ^£ presclio©! ej«peri€-nce' 
prior t© enteriftf Head start 1970, 

fi££ "^"^ v-sriabi© iftdicmiftf a (sfeild's age ir* 

'^'^'^ Teaaier CJ5®rac£©risfciCS variafcies t^en fmm 

the teathtf irsforp-a tiers forro «sc-c Ch&ptm tlU 

Rtiifiscr of f€:drs ©f exp&ea©r.ce tliat the tesms:fsQr has 
in Head Stas't prior co Ifie. ^ 

wi»tli€?r er not a teactier fc^as certified fev tis© 
City er state"'*.© temt ir* a pf©-seiJe©I @r pimlifs 

vari aisle ifisieaSkjBg wl.etli«r tte 
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peasure %&a&mm* c-val nations of their worHinf 

This variable- t?as tsken frfjm the Teactsei; Md© 
qyastionr^airc^ a.rid assesi.es tfoe ny^er r^f years of ^^cperic^iice 

/ 

Tiiis vari.aple ^as taken fmm m Head Stars Director's 
sfcryet«re of tf»4> Head Start Center °- %?riCtAer' a 

the 'ptifeli.e s€:'P&i55iG, 

2, "'Site is eitlier afi eptefirscj fa/jfst ©r c-iitc^riRfi 
kinder gar ten Thas variable? t:Afm ftmi 

tirvi!^ 5=is-5»de ©ff kandcfgcaftiGfj di'rc>c?tly after Iteae m&fft^ 



